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Appendices 


Annual  Summary  Report 


Award  Number:  DAMD1 7-02-1 -0230 

FY01  Prostate  Cancer  Research  Program  (PCRP)  Post-Doctoral  Traineeship  Award 
"Molecular  Imaging  for  IMRI-Guided  Minimally  Invasive  Treatment  of  Prostate  Cancer" 


Introduction 

Our  long-term  goal  is  to  develop  an  image-guided,  minimally  invasive  treatment  method  for  prostate 
cancer.  We  currently  use  interventional  magnetic  resonance  imaging  (iMRI)  to  guide  biopsy  needles  for 
diagnosis  and  to  guide  radiofrequency  (RF)  electrodes  for  thermal  ablation  treatment  of  cancer.  We  use  a 
low  field,  open  magnet  system  that  allows  patient  access  during  imaging.  A  unique  aspect  of  iMRI-guided 
thermal  ablation  therapy  is  that  real  time  feedback  is  provided  from  images  of  treated  regions  and  from 
non-invasive  three-dimensional  (3D)  temperature  measurements.  Before  this  method  can  be  extended  to 
the  prostate,  significant  engineering  developments  are  required. 

Although  MRI  provides  excellent  anatomical  detail,  it  does  not  reliably  identify  prostate  cancer. 
Fortunately,  there  are  emerging  imaging  methods  that  promise  to  improve  detection  of  prostate  tumor. 
These  include  nuclear  medicine  SPECT  and  MR  spectroscopy  that  promise  to  help  identify  prostate 
tumor.  We  intend  to  incorporate  this  information  especially  SPECT  ProstaScint  images  into  the  iMRI- 
guided  paradigm. 

Although  functional  images  promise  to  help  identify  prostate  tumor,  they  have  poor  resolution  and  do 
not  provide  the  anatomical  information  required  for  treatment  delivery.  The  solution  is  to  register  and 
superimpose  the  functional  images  with  3D  high-resolution  MR  images.  We  are  investigating  method  to 
correlate  features  from  SPECT  with  high-resolution  anatomical  MR  images  to  aid  identification  of 
prostate  cancer.  To  improve  tumor  targeting  during  treatment,  the  information  from  the  multi-modality 
will  be  integrated  with  freshly  acquired  iMRI-slice  image  to  guide  intervention  tools.  Accurate  registration 
with  anatomy  will  be  necessary  for  highly  accurate,  interventional  MRI-guided  thermal  ablations. 

Summary  Body 

As  part  of  my  postdoctoral  training,  I  am  working  with  a  team  of  researchers  to  achieve  aims.  My 
specialized  role  is  to  develop  image  registration  techniques,  but  I  participated  in  most,  if  not  all,  research 
aspects  of  the  molecular  imaging  for  iMRI-guided  minimally  invasive  treatment  of  prostate  cancer.  1-1 1 
The  following  paragraphs  outline  the  training  and  research  associated  with  the  tasks  described  in  the 
Statement  of  Work. 

As  for  the  Task  1  of  image  acquisitions,  we  obtained  and  analyzed  SPECT  ProstaScint  and  high- 
resolution  MR  images  from  patients  with  verifiable  prostate  cancer.12'13  First,  we  developed  methods  for 
SPECT  ProstaScint  imaging  for  the  prostate  cancer.13-18  Second,  we  developed  high-resolution  3D  MR 
imaging  techniques  for  the  prostate.1-3'7  The  imaging  techniques  are  reported  in  our  published  papers.1"3  7 1 
helped  plan  image  acquisition  methods  suitable  for  registration. 

As  for  the  Task  2  of  image  correlation  and  registration,  we  have  correlated  SPECT  images  with  high- 
resolution  MRI  volume  to  improve  detection  and  treatment  of  prostate  cancer.6  7,13'19'20  First,  we  created 
3D  image  registration  methods  suitable  for  aligning  multi-modality  volumes  to  combine  anatomic  and 
metabolic  information.6’7’19  We  extended  our  current  mutual  information  methods  for  robust  image 
registration.  Second,  we  created  image  software  system  to  visualize  and  analyze  images  of  the  prostate 
cancer.1'6'7  Using  the  visualization  software,  we  compared  3D  registered  image  data  using  linked  cursors 
and  hand-drawn  regions-of-interest  on  side-by-side  image  frames.  From  a  pair  of  registered  volumes,  we 
applied  multi-planar  reformatting  to  visualize  registered  slices  from  each  of  two  volumes  using  multiple 
types  of  displays.  Statistical  analyses  were  developed  to  correlate  image  gray  values  in  ProstaScint  and 
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MRI  images.13'17'18  In  addition  to  analysis,  we  investigated  fusion  methods  for  iMRI  targeting.  These 
techniques  are  described  in  detail  in  our  published  papers.1'6'7  Third,  our  group  compared  functional  image 
data  with  histology  of  surgically  removed  prostates.12'13 

As  for  the  Task  3  of  testing  the  feasibility  of  iMRI-guided  procedure,  we  have  performed  simulation 
experiments,6  phantom  experiments,6'7  iMRI-guided  thermal  ablation  experiments  on  dogs  and  rabbits,21'23 
and  simulated  clinical  experiments  using  patient  data.6'7  First,  we  created  a  robust,  accurate  and  fast 
registration  method  for  freshly  acquired  iMRI-slice  images  and  SPECT/MR  volume  images.1'3'11  We 
performed  registration  experiments  at  our  low  field,  open  magnetic  system.1'3'11  Second,  we  conducted 
phantom  experiments  and  evaluated  the  registration  accuracy.  1,6,7  Third,  we  have  performed  simulation 
experiments  for  the  image-guided  thermal  ablation  procedure  using  patient  data.1,6,7  We  have  conducted 
thermal  ablation  experiments  on  dogs  and  rabbits.21'23 

As  for  the  training,  I  actively  participated  in  the  journal  club  every  week  as  held  in  Dr.  Wilson’s 
Laboratory.  I  attended  the  research  seminar  once  a  week  in  the  Biomedical  Engineering  Department.  The 
Ireland  Cancer  Center  at  our  institute,  a  NCI  Comprehensive  Cancer  Center  with  over  130  scientists  and 
physicians,  often  has  excellent  seminars  with  great  training  opportunities.  I  frequently  interact  with 
mentors  Drs.  Wilson,  Duerk,  and  Sodee.  I  took  eight  courses  in  biomedical  engineering  and  computer 
science  at  Case  Western  Reserve  University.  I  had  nine  international  conference  presentations  during  the 
two  years. 

Key  Research  Accomplishments 

1 .  We  created  a  robust  image  registration  method  for  the  combination  of  high  resolution  MRI  volume 
and  interventional  MRI  slice.1,3'8'11  It  is  significant  because  it  demonstrated  feasibility  of  including 
other  image  modalities  such  as  high  resolution  MRI  and  SPECT  images  into  iMRI-guided  treatment 
for  improved  tumor  targeting. 

2.  We  created  image  registration  and  fusion  software  that  could  be  used  for  the  iMRI-guided 
procedures.6'7  It  is  important  because  it  could  help  physician  or  radiologist  better  locate  and  target 
tumors  region  and  possibly  improve  treatment. 

3.  We  developed  MR  imaging  techniques  for  the  acquisition  of  high  resolution  prostate  images.1"3  Since 
the  imaging  techniques  well  delineates  the  prostate  and  its  surroundings,  they  could  possibly  be  used 
for  the  prostate  cancer  screening  and  other  clinical  applications. 

4.  We  developed  fast  imaging  techniques  for  interventional  MR  imaging  of  the  prostate.1,3  This  is  very 
important  because  the  techniques  can  acquire  prostate  images  in  nearly  real-time  and  thus  provide 
image  guidance  during  interventional  procedures. 

5.  We  created  image  visualization  software  for  image  analysis  and  registration  evaluation.6'7  This 
byproduct  is  a  very  useful  image  analysis  tool  with  multiple  visualization  displays.  It  also  allows  one 
to  quantitatively  measure  prostate  size,  movement,  and  deformation. 

6.  We  performed  more  than  1000  slice  to  volume  registration  experiments  using  MR  images  acquired 
from  patients  and  volunteers.1,3'8'11  These  experiments  simulated  and  tested  a  variety  of  possible 
conditions  in  potential  applications  and  thus  provide  useful  information  and  guidance  for  future 
clinical  applications. 

7.  We  performed  image  registration  and  fusion  experiments  using  SPECT  and  high  resolution  MR 
images  from  patients  with  prostate  cancer.  Registration  and  fusion  of  the  two  modalities  could 
possible  improve  the  diagnosis  of  prostate  tumors.6'7'19 

8.  We  conducted  a  variety  of  pelvic  phantom  experiments  to  test  registration  accuracy  of  high  resolution 
MR  and  SPECT  images.6,7'19  These  experiments  provided  some  baseline  for  accuracy  evaluation  and 
simulation. 

9.  We  conducted  iMRI-guided  thermal  ablation  experiments  on  dogs  and  rabbits.21'23  The  experiments 
provided  some  fundamental  knowledge  for  any  future  human  applications. 
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10.  We  have  applied  the  volume-to-volume  registration  technique  to  vascular  images  for  the  potential 
application  in  vulnerable  plaque  classification.24'25 

11.  We  have  applied  our  volume-to-volume  registration  algorithm  to  micro-PET  and  high-resolution  MR 
images  for  the  potential  application  in  photodynamic  therapy.26 

12.  We  have  applied  our  volume-to-volume  registration  method  to  small  animal  images  as  acquired  from 
micro-CT  and  SPECT  for  the  application  in  lung  perfusion  study.27 

Reportable  Outcomes 

1 .  Our  paper  entitled  “Slice  to  volume  registration  and  its  potential  application  to  interventional  MRI 
guided  radiofrequency  thermal  ablation  of  prostate  cancer”1  was  published  in  the  IEEE  Transaction  on 
Medical  Imaging,  which  was  the  most-cited  journal  in  the  biomedical  engineering  and  imaging 
science  &  photographic  technology  categories.28 

2.  One  paper  entitled  “Automatic  3D  registration  for  interventional  MRI-guided  treatment  of  prostate 
cancer”3  was  published  in  the  Computer  Aided  Surgery,  the  official  journal  of  the  International 
Society  for  Computer  Aided  Surgery  (ISCAS).29 

3.  One  paper  entitled  “A  comparative  study  of  warping  and  rigid  body  registration  for  the  prostate  and 
pelvic  MR  volumes”4  was  published  in  the  Computerized  Medical  Imaging  and  Graphics,  the  official 
journal  of  the  Computerized  Medical  Imaging  Society,  an  international  journal  on  imaging  and  image¬ 
archiving  in  all  medical  specialties.30 

4.  One  paper  entitled  “Registration  and  fusion  of  SPECT,  high  resolution  MRI,  and  interventional  MRI 
for  thermal  ablation  of  prostate  cancer”7  is  published  in  IEEE  Transaction  on  Nuclear  Science. 

5.  One  manuscript  entitled  “Three-dimensional  MR  image  registration  of  the  prostate  and  pelvis  with 
semiautomatic  warping”31  was  submitted  to  IEEE  Transaction  on  Medical  Imaging  in  March  2004. 

6.  Two  referred  conference  papers  were  published  in  the  Lecture  Notes  in  Computer  Science:  the 
proceeding  of  the  Sixth  International  Conference  on  Medical  Image  Computing  and  Computer 
Assisted  Intervention  2003  6  and  the  proceeding  of  the  Second  International  Workshop  on  Biomedical 
Image  Registration.5 

7.  Three  conference  papers  were  published  in  the  proceedings  of  International  Conference  of  SPIE  on 
Medical  Imaging  2002,  2003,  and  2004.9'10'26 

8.  Three  conference  papers  were  published  in  the  proceedings  of  the  International  Conference  of  IEEE 
Engineering  in  Medicine  and  Biology  2002  and  2003. 8'24'27 

9.  I  had  five  oral  presentations  and  four  poster  presentations  at  professional  international  conferences: 
The  24th  and  25th  IEEE  Engineering  in  Medicine  and  Biology,8'24'27  The  SPIE  on  Medical  Imaging 
Conferences  (2002,  2003,  and  2004), 91 126  The  Sixth  International  Conference  on  Medical  Image 
Computing  and  Computer  Assisted  Intervention  2003,6  the  Second  International  Workshop  on 
Biomedical  Image  Registration,5  and  The  10th  Scientific  Retreat  of  the  Prostate  Cancer  Foundation.26 

10.  I  wrote  two  book  chapters  for  the  Handbook  of  Medical  Image  Analysis:  Advanced  Segmentation  and 
Registration  Models.32'33 

Conclusions 

During  the  two  year  of  training,  we  well  completed  the  tasks  as  outlined  in  the  Statement  of  Work.  We 

achieved  twelve  key  research  accomplishments  and  ten  reportable  outcomes.  We  had  seven  peer-reviewed 

referred  papers,  seven  conference  proceedings,  and  nine  scientific  presentations.  We  achieved  the  goals  of 

the  training  and  research. 
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Abstract — In  this  study,  we  registered  live-time  interventional 
magnetic  resonance  imaging  (iMRI)  slices  with  a  previously 
obtained  high-resolution  MRI  volume  that  in  turn  can  be  reg¬ 
istered  with  a  variety  of  functional  images,  e.g.,  PET,  SPECT, 
for  tumor  targeting.  We  created  and  evaluated  a  slice-to-volume 
(SV)  registration  algorithm  with  special  features  for  its  potential 
use  in  iMRI-guided  radio-frequency  (RF)  thermal  ablation  of 
prostate  cancer.  The  algorithm  features  included  a  multiresolution 
approach,  two  similarity  measures,  and  automatic  restarting 
to  avoid  local  minima.  Imaging  experiments  were  performed 
on  volunteers  using  a  conventional  1.5-T  MR  scanner  and  a 
clinical  0.2-T  C-arm  iMRI  system  under  realistic  conditions.  Both 
high-resolution  MR  volumes  and  actual  iMRI  image  slices  were 
acquired  from  the  same  volunteers.  Actual  and  simulated  iMRI 
images  were  used  to  test  the  dependence  of  SV  registration  on 
image  noise,  receive  coil  inhomogeneity,  and  RF  needle  artifacts. 
To  quantitatively  assess  registration,  we  calculated  the  mean  voxel 
displacement  over  a  volume  of  interest  between  SV  registration 
and  volume-to-volume  registration,  which  was  previously  shown 
to  be  quite  accurate.  More  than  800  registration  experiments  were 
performed.  For  transverse  image  slices  covering  the  prostate,  the 
SV  registration  algorithm  was  100%  successful  with  an  error 
of  <2  mm,  and  the  average  and  standard  deviation  was  only 
0.4  mm  ±  0.2  mm.  Visualizations  such  as  combined  sector  display 
and  contour  overlay  showed  excellent  registration  of  the  prostate 
and  other  organs  throughout  the  pelvis.  Error  was  greater  when 
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an  image  slice  was  obtained  at  other  orientations  and  positions, 
mostly  because  of  inconsistent  image  content  such  as  that  from 
variable  rectal  and  bladder  filling.  These  preliminary  experiments 
indicate  that  MR  SV  registration  is  sufficiently  accurate  to  aid 
image-guided  therapy. 

Index  Terms — Image  registration,  interventional  magnetic  reso¬ 
nance  imaging  (iMRI),  minimally  invasive  treatment,  mutual  in¬ 
formation,  prostate  cancer,  thermal  ablation. 

I.  Introduction 

WE  USE  AN  interventional  magnetic  resonance  imaging 
(iMRI)  system  to  guide  minimally  invasive  treatments, 
including  the  radio-frequency  (RF)  thermal  ablation  of  abdom¬ 
inal  cancers  [l]-[3].  The  iMRI  system  consists  of  a  0.2-T  clin¬ 
ical  C-arm  open  MRI  scanner,  an  in-room  RF-shielded  liquid 
crystal  monitor,  an  MR  compatible  mouse,  a  foot  pedal,  and 
an  RF  device.  We  are  currently  investigating  the  extension  of 
these  techniques  to  the  treatment  of  prostate  cancer.  Since  MRI 
does  not  reliably  show  prostate  tumors,  we  intend  to  incorpo¬ 
rate  nuclear  medicine  or  MR  spectroscopy  images  with  higher 
sensitivity  for  detecting  and  localizing  prostate  tumors  [4],  [5]. 
We  will  first  register  the  low-resolution  functional  images  with 
a  high-resolution  MRI  volume  [6],  [7].  Then,  by  registering  the 
high-resolution  MR  volume  with  live-time  iMRI  acquisitions, 
we  can,  in  turn,  map  the  functional  data  and  high-resolution 
anatomic  information  to  live-time  iMRI  images  for  improved 
tumor  targeting.  As  discussed  later,  since  live-time  iMRI  is  used 
for  device  guidance,  the  accuracy  requirements  for  registering 
these  supplemental  images  might  be  less  strict  than  required  in 
some  other  applications. 

We  previously  described  a  rigid-body  volume-to-volume 
( V V)  registration  method  for  the  pelvic  and  prostate  MR  images 
that  was  accurate  when  images  were  acquired  under  similar 
conditions  [8].  We  used  bony  landmarks  and  three-dimensional 
(3-D)  centroids  of  segmented  prostates  to  evaluate  VV  reg¬ 
istration.  For  volume  pairs  acquired  over  a  short  time  span 
from  a  supine  subject  with  legs  flat  on  the  table,  registration 
accuracy  of  both  the  prostate  centroid  (typically  <1  mm) 
and  bony  landmarks  (average  1.6  mm)  was  on  the  order  of 
a  voxel  (wl.4  mm).  The  centroid  error  was  slightly  smaller 
because  the  prostate  was  at  the  volume  center  and  rotation 
errors  had  less  effect  on  it.  The  localization  error  in  finding 
3-D  points  from  bony  landmarks  is  probably  greater  than  that 
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of  finding  centroids  of  relatively  large  prostate  volumes  where 
segmentation  errors  average  out.  We  obtained  somewhat  larger 
prostate  registration  errors  of  about  3.0  mm  when  volume  pairs 
were  obtained  under  very  different  conditions  that  would  be 
avoided  in  patient  studies,  e.g.,  legs  flat  and  legs  raised. 

In  this  study,  we  are  investigating  methods  to  register 
live-time  iMRI  image  slices  with  a  previously  obtained 
high-resolution  MRI  volume.  We  call  this  slice-to-volume 
(SV)  registration.  Because  of  our  success  with  VV  prostate 
registration,  we  can  determine  SV  accuracy  by  comparing 
results  to  VV  registration  for  volume  pairs  having  low  VV 
registration  error. 

The  application  of  SV  registration  to  iMRI-guided  treatment 
of  prostate  cancer  raises  several  challenges.  First,  a  single  slice, 
or  a  few  slices,  provides  much  less  information  than  an  entire 
volume  for  voxel-based  matching.  Second,  iMRI  images  often 
have  lower  signal-to-noise  ratio  (SNR)  than  diagnostic  MR  im¬ 
ages  because  of  the  emphasis  on  fast  imaging  and  because  of 
the  typically  lower  field  strength  of  open  iMRI  magnets.  Third, 
the  normal  prostate  is  a  small  organ;  when  healthy,  it  measures 
only  ^3.8  cm  in  its  widest  dimension  [9].  The  small  prostate  is 
located  below  the  much  larger  bladder  that  can  change  its  shape 
and  size  during  imaging.  Fourth,  the  nonhomogenous  receive 
coil  response  can  change  from  one  imaging  session  to  the  next. 
Finally,  times  for  registration  and  algorithm  robustness  are  of 
particular  concern  for  this  application  to  treatment. 

Previously  reported  methods  for  SV  registration  were  mainly 
applied  to  the  brain  for  applications  of  functional  MRI  [10], 
postmortem  pathology  studies  [11],  and  anatomical  modeling 
[12].  There  are  no  reports  of  SV  registration  for  abdominal  or¬ 
gans  or  iMRI  guidance.  Voxel-based  methods,  particularly  those 
based  upon  mutual  information  (MI),  are  robust,  require  no  seg¬ 
mentation  that  can  be  prone  to  error,  are  suitable  for  multi¬ 
modality  registration,  and  are  highly  accurate  for  many  appli¬ 
cations  [3],  [8],  [10],  [13]— [15].  However,  the  MI  method  has 
the  problem  of  interpolation  artifacts,  which  can  be  especially 
serious  in  the  case  of  downsampling  in  a  multiresolution  ap¬ 
proach  [16].  Other  similarity  measures  such  as  the  correlation 
coefficient  (CC)  can  reduce  the  presence  of  local  minima  [17]. 

In  this  paper,  we  first  describe  a  voxel-based  registration  al¬ 
gorithm  with  special  features  for  this  important  new  applica¬ 
tion.  Later,  we  describe  the  details  of  imaging  experiments  on 
a  conventional  MR  scanner  and  a  clinical  iMRI  system.  Ac¬ 
tual  and  simulated  iMRI  images  are  used  to  test  the  registra¬ 
tion  algorithm.  Results  of  S  V  and  VV  registration  are  compared. 
In  this  study,  we  have  performed  more  than  800  registration 
experiments. 


II.  Registration  Algorithm 
A.  Similarity  Measurements 

We  used  two  similarity  measures — mutual  information  and 
correlation  coefficient — in  our  registration.  Suppose  one  image 
R  is  the  reference ,  and  the  other  F  is  floating.  Their  mutual 
information  MI(i2,  F)  is  given  as  follows  [18]: 


MI(i?,F)  =  y>RP(r,/)log 

rj 


'PRFjrJ) 
pr{7’)  •  pF(f) ' 


The  joint  probability  prf(t,  /)  and  the  marginal  probabilities 
Pr(v)  of  the  reference  image  and  pr(f)  of  the  floating  image 
can  be  estimated  from  the  normalized  joint  intensity  histogram. 
The  correlation  coefficient  CC(J2,  F)  is  given  as  follows  [19]: 

CC(R,F)  =  E (KM- KM)  (W) -?(/)) 

\Ze(kM-kM)2E(K(/)-k(/))2' 

Here  /2(r),  F(f )  denote  the  average  intensities  of  the  refer¬ 
ence  and  floating  images  and  the  summation  includes  all  voxels 
within  the  overlap  of  both  images. 

We  compared  the  two  similarity  measures  at  different  reso¬ 
lutions  in  order  to  determine  their  suitability  for  SV  registra¬ 
tion.  At  1/4  resolution ,  we  resampled  images  so  as  to  give  1/4 
number  of  the  voxels  along  each  linear  dimension.  At  full  reso¬ 
lution ,  we  used  the  full  number  of  voxels.  In  Figs.  1  and  2,  we 
plot  the  two  similarity  measures  as  a  function  of  two  transla¬ 
tion  parameters.  After  two  typical  high-resolution  MR  volumes 
were  registered  [8],  values  were  plotted  with  the  origin  as  the 
optimal  transformation.  We  calculated  CC  and  MI  values  while 
moving  the  simulated  iMRI  image  relative  to  the  high-resolu- 
tion  MR  image  along  coronal  (anterior-posterior)  and  sagittal 
(left-right)  axes.  The  simulated  iMRI  image  was  obtained  as 
described  in  Section  III. 

Features  of  MI  and  CC  demonstrate  their  suitability  at  high 
and  low  resolutions,  respectively.  At  1/4  resolution,  CC  sur¬ 
faces  are  much  smoother  than  MI,  which  is  noisy  and  contains 
many  local  maxima  as  shown  in  Fig.  1(a)  and  (c).  In  fact,  there 
is  a  false  global  maximum  at  +25  voxels.  At  full  resolution, 
Fig.  2(a)  and  (c)  shows  that  MI  has  a  much  sharper  peak  than 
CC,  but  once  again  there  is  high-frequency  noise  in  the  MI 
curves,  far  from  the  optimum,  that  gives  rise  to  local  maxima 
that  must  be  avoided.  From  these  figures,  we  infer  that  CC  is 
better  at  low  resolution  and  MI  is  better  at  full  resolution,  when 
one  is  close  to  the  optimum  value.  As  described  next,  our  regis¬ 
tration  algorithm  makes  use  of  these  features. 

B.  Registration  Algorithm  with  Special  Features 

The  algorithm  includes  special  features  to  improve  robust¬ 
ness  for  registration  of  MR  prostate  images.  Suppose  the  iMRI 
image  slice  is  the  reference  slice ,  the  matching  slice  extracted 
from  the  high-resolution  MRI  volume  is  the  reformatted  slice , 
and  the  final  reformatted  slice  is  the  registered  slice.  We  use 
a  multiresolution  approach  and  perform  registration  from  low 
to  high  resolution.  We  use  CC  at  the  two  lower  resolutions  be¬ 
cause  it  gives  fewer  local  maxima  and  because  it  can  be  calcu¬ 
lated  faster  than  MI.  We  use  MI  at  full  resolution  because  of  its 
peaked  surface.  To  avoid  local  maxima,  we  include  a  restarting 
feature  where  registration  is  restarted  with  randomly  perturbed 
parameters  obtained  from  a  uniform  distribution  about  the  ini¬ 
tial  transformation  values  at  the  current  resolution  being  used. 
The  algorithm  restarts  until  the  absolute  CC  is  above  a  threshold 
of  0.5  as  determined  later  or  the  maximum  number  of  restarts 
is  reached.  Absolute  CC  is  used  rather  than  MI  because  it  has  a 
well-defined  range  between  0  and  1  and  because  it  provides  an 
independent  check  of  the  MI  result  at  the  highest  resolution. 

We  record  all  important  results  following  an  optimization 
cycle  including  the  CC  and/or  MI  values  and  the  transforma- 
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(a) 


(b) 


(c)  (d) 

Fig.  1.  Similarity  functions  are  plotted  as  a  function  of  translations  at  the  lowest  resolution  in  the  multiresolution  registration  process.  Two  high-resolution  MRI 
volumes  were  registered.  From  the  optimal  parameters,  we  computed  the  similarity  of  the  simulated  iMRI  and  MRI  images  as  a  function  of  translations  along  the 
coronal  (anterior-posterior)  and  sagittal  (left-right)  axes.  MI  is  plotted  in  (a)  and  (c);  CC  is  plotted  in  (b)  and  (d).  Graphs  (a)  and  (b)  are  3-D  plots  for  translations 
along  the  coronal  and  sagittal  axis.  Graphs  (c)  and  (d)  are  2-D  plots  for  translations  about  the  coronal  axis.  The  small  insets  in  (c)  and  (d)  are  magnified  curves 
showing  noise  having  local  maxima  in  (c).  A  false  global  maximum  for  MI  occurred  at  -f-25  voxels.  Images  are  from  volunteer  S2,  and  they  are  downsampled  by 
1/4  along  each  linear  dimension,  giving  a  distance  between  voxel  centers  of  «5.5  mm. 


tion  parameters.  At  the  end  of  processing  at  a  lower  resolution, 
we  always  select  the  transformation  parameters  having  the  max¬ 
imum  CC  value.  We  then  scale  the  translation  parameters  appro¬ 
priately  and  assign  the  new  parameters  to  be  initial  values  at  the 
next  higher  resolution.  At  the  highest  resolution,  MI  instead  of 
CC  is  the  similarity  measure,  and  we  select  the  final  transfor¬ 
mation  parameters  to  be  those  having  the  maximum  MI  value. 

C.  Additional  Details 

Additional  algorithm  details  are  now  described.  For  registra¬ 
tion,  we  use  rigid-body  transformation  (three  translations  and 
three  rotations)  and  trilinear  interpolation.  For  optimization,  we 
use  the  downhill  simplex  method  of  Nelder  and  Mead  [20]  or 
the  Powell  method  [21],  Optimization  of  similarity  ends  either 
when  the  maximum  number  of  calculations  is  reached  (typically 
500)  or  the  fractional  change  in  the  similarity  function  is  smaller 
than  a  tolerance  (typically  0.001).  The  input  MRI  volume  is 
a  3-D  MR  acquisition  giving  256  x  256  x  128  nearly  isotropic 


voxels  over  a  field  of  view  covering  the  whole  pelvis.  We  create 
isotropic  voxels  of  about  1.4  mm  on  a  side  using  3-D  linear  in¬ 
terpolation.  We  use  IDL  (Interactive  Data  Language,  Research 
Systems  Inc.,  Boulder,  CO)  as  the  programming  language. 

Typical  parameter  values  are  now  described.  We  use  an  initial 
guess  assuming  an  identity  transformation,  i.e.,  all  initial  trans¬ 
lation  and  rotation  parameters  are  zero,  because  the  patient  is 
normally  oriented  approximately  the  same  way  from  one  scan 
to  the  next.  We  set  the  maximum  numbers  of  restarts  at  10,  5, 
and  3,  from  low  to  high  resolution,  respectively. 

III.  Experimental  Methods 
A.  Image  Acquisition 

High-resolution  MRI  volumes  were  acquired  using  a  1.5-T 
Siemens  MRI  system  (Magnetom  Symphony,  Siemens  Medical 
Systems,  Erlangen,  Germany).  An  eight-element  phased  array 
body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a 


518 


IEEE  TRANSACTIONS  ON  MEDICAL  IMAGING,  VOL.  22,  NO.  4,  APRIL  2003 


(c)  (d) 

Fig.  2.  Similarity  functions  are  plotted  as  a  function  of  translations  at  full  resolution.  Many  details  are  given  in  the  legend  of  Fig.  1.  Again,  MI  is  plotted  in  (a) 
and  (c);  CC  is  plotted  in  (b)  and  (d).  MI  in  (a)  and  (c)  has  a  much  sharper  peak  than  CC  in  (b)  and  (d).  The  voxel  is  isotropic  with  1.4  mm  on  a  side.  Image  data 
are  the  same  used  in  Fig.  I . 


uniform  sensitivity.  Typically,  two  anterior  and  two  posterior 
elements  were  enabled  for  signal  acquisition.  We  used  two  dif¬ 
ferent  MR  sequences.  First,  we  used  a  3-D  RF  spoiled  gradient 
echo  steady-state  pulse  sequence  (FLASH)  with  TR/TE/flip 
parameters  of  12/5.0/60,  which  give  256  x  256  x  128  voxels 
over  a  330  x  330  x  256-mm  field  of  view  (FOV)  to  yield 
1.3  x  1.3  x  2.0-mm  voxels  oriented  to  give  the  highest  resolu¬ 
tion  for  transverse  slices.  The  acquisition  time  is  5  min  and  38 
s.  This  sequence  is  good  for  pelvic  imaging,  but  is  not  ideal  for 
the  prostate.  It  was  used  to  acquire  volumes  for  volunteer  SI. 
Second,  we  used  a  3-D  rapid  gradient  echo  sequence  (PSIF) 
designed  to  acquire  the  spin-echo  component  of  the  steady-state 
response,  rather  than  the  free  induction  decay.  The  spin  echo 
component  forms  immediately  prior  to  the  RF  pulse;  it  is 
shifted  toward  the  prior  RF  pulse  through  appropriate  gradient 
waveform  design.  The  sequence  with  9.4/5.0/60  (TR/TE/flip) 
yields  160  X  256  x  128  voxels  over  a  219  x  350  x  192-mm 
rectangular  FOV  and  1.4  x  1.4  x  1.5-mm  voxels  oriented 
to  give  the  highest  resolution  for  transverse  slices.  There  is 
over  sampling  at  31%  in  the  slice  direction  to  reduce  aliasing 
artifacts.  The  acquisition  time  is  4  min  and  15  s.  This  sequence 


gave  excellent  image  contrast  for  the  prostate  and  its  surround¬ 
ings.  It  was  used  to  acquire  volumes  for  volunteers  S2-S4. 

We  also  acquired  iMRI  images  from  the  same  volunteers 
using  a  clinical  0.2-T  C-arm  open  MR  scanner  (Siemens  Open 
Symphony,  Erlangen,  Germany)  modified  for  interventional 
MRI  procedures  and  in  this  paper  referred  to  as  the  iMRI 
system.  We  used  a  3-D  PSIF  with  25/13/60  (TR/TE/FA)  for 
image  volume  acquisitions  and  two-dimensional  (2-D)  PSIF 
with  15.2/7.4/45  (TR/TE/FA)  for  image  slice  acquisitions. 
The  iMRI  volumes  were  256  x  256  x  100  with  voxel  size  of 
1.3  x  1.3  x  1.4  mm.  The  iMRI  slices  were  128  x  128  with 
in-plane  pixel  size  of  2.8  x  2.8  mm  and  with  effective  slice 
thickness  of  5  mm.  We  acquired  iMRI  images  from  volunteers 
S1-S3. 

B.  Simulation  of  iMRI  Image  Slices 

In  experiments,  we  used  high-resolution  MRI  volumes  to 
simulate  iMRI  image  slices,  which  are  thicker,  noisier,  and  de¬ 
graded  by  receive  coil  inhomogeneity.  Clinically,  we  typically 
use  an  iMRI  slice  thickness  of  4.0-6.0  mm.  We  used  trilinear 
interpolation  to  create  isotropic  high-resolution  MRI  volumes 
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Fig.  3.  High-resolution  MR  images,  simulated  and  actual  iMRI  image  slices.  Images  on  the  left  column,  (a),  (d),  and  (g),  are  the  original  high-resolution  MR 
images  from  the  1.5-T  scanner  in  the  transverse,  coronal,  and  sagittal  planes,  respectively.  Images  in  the  middle  column  are  the  corresponding,  simulated  thick 
iMRI  images  with  noise  added  to  give  SNR  =  15  and  with  sensitivity  fall  off  from  a  belt  coil.  Images  on  the  right  panel  are  actual  iMRI  slices  (0.2-T  scanner) 
from  similar  spatial  locations.  The  actual  iMRI  slices  seem  blurred  because  of  nearly  doubled  pixel  size.  Images  are  from  volunteer  S2. 


with  voxel  size  of  1.4  x  1.4  x  1.4  mm.  From  the  isotropic 
high-resolution  MRI  volume,  we  averaged  three  1.4-mm 
adjacent  thin  slices  to  create  a  4.2-mm-thick  slice.  MR  noise 
in  a  magnitude  is  described  by  the  Rician  distribution  [22].  At 
SNR  values  of  greater  than  approximately  five,  the  noise  can  be 
approximated  as  being  Gaussian  white  noise  [23].  We  measured 
typical  signal  and  noise  values  on  our  iMRI  system  using  a 
homogenous  phantom,  and  volunteer  images  in  the  region  of 
the  prostate  with  methods  described  elsewhere  [24],  [25].  In  all 
cases,  image  SNR  was  greater  than  10  in  all  tissues  including 
the  prostate.  With  this  justification,  we  added  Gaussian  noise  to 
the  simulated  iMRI  image  slices  either  to  match  the  measured 
SNR  or  to  give  much  greater  noise  to  further  stress  registration. 
We  report  noise  experiments  using  the  SNR  of  the  simulated 
image  slices.  Fig.  3  shows  high-resolution  MR  images  as  well 
as  simulated  and  actual  iMRI  image  slices. 

We  simulated  receive  coil  inhomogeneity  from  a  belt  coil 
used  in  our  clinical  iMRI  acquisitions.  The  coil  is  modeled  as  a 
solenoid  with  parameters  shown  in  Fig.  4.  Coil  parameters  are 
a,  the  radius  of  the  coil;  2  g,  the  length  of  the  coil;  /,  the  current; 
Ho,  the  permeability  of  free  space;  n,  the  turns;  and  the  z  axis, 
the  axis  along  the  center  line  of  the  coil.  The  magnetic  field  in 
the  xy  plane  can  be  approximated  as  [26] 


(51  )xy 


fipn  / 

2  [a2  4-  P2]1/Z 


(1) 


The  2:  component  of  the  field  is  given  by  [27] 


(Bx)z  =  (cos  ai  +  cos  a2)  (2) 

where  the  definition  of  the  angles  ot\  and  a 2  are  given  in  Fig.  4. 
The  magnetic  field  is  highest  at  the  coil  center  and  falls  off  along 
the  axial  direction.  According  to  the  Biot-Savart  law  [28],  this 
model  also  accounts  for  the  spatial  sensitivity  of  the  coil  to  MR 


Fig.  4.  Geometry  of  solenoidal  receive  coil.  Model  parameters  are  defined  in 
the  figure.  The  axial  line  is  along  the  cranial-caudal  direction  of  the  patient. 

signal  sources.  Fig.  5  shows  a  coronal  image  with  simulated 
inhomogeneity  along  the  axis  (head-foot)  direction. 

Because  a  needle  will  often  be  present  during  an  iMRI  in¬ 
tervention,  we  tested  the  effect  of  simulated  needles  on  regis¬ 
tration.  We  used  artifact  sizes  from  a  previous  report  on  the  ef¬ 
fects  of  pulse  sequence  design  and  magnetic  field  orientation  on 
needle  artifacts  in  MR- guided  biopsy  and  aspiration  [29].  Fig.  6 
shows  sagittal  images  with  and  without  a  simulated  needle  ar¬ 
tifact.  The  simulated  artifacts  in  Fig.  6(b)  appeared  as  straight 
noisy  bars  2  mm  in  width. 

C.  Imaging  Experiments 

1 )  Imaging  Experiments  for  High-Resolution  MR 
Volumes :  When  acquiring  high-resolution  MR  volumes, 
volunteers  laid  supine  in  a  manner  similar  to  the  diagnostic 
position  in  routine  MR  scanning.  Between  volume  acquisitions, 
volunteers  got  up  from  the  MR  table,  stretched,  and  walked 
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Fig.  5.  Simulated  signal  changes  due  to  receive  coil  inhomogeneity.  The 
original  image  (a)  is  acquired  using  a  phased  array  coil  on  a  conventional  1.5-T 
MRI  system.  Using  a  belt  coil  model  with  a  diameter  of  350  mm  and  a  width 
of  50  mm,  the  simulated  iMRI  image  is  shown  in  (b).  The  image  intensity  is 
highest  at  the  center  and  decreases  along  the  axial  direction. 


Fig.  6.  Synthetic  image  with  simulated  needle  artifact.  Image  (a)  is  the  sagittal 
slice  acquired  from  the  0.2  T  iMRI  system  without  a  needle  artifact.  Image  (b) 
is  obtained  from  image  (a)  with  a  simulated  needle  artifact  (white  arrow)  for  an 
RF  needle  probe  inserted  into  the  prostate.  Images  are  from  volunteer  S3. 

around  to  ensure  that  they  would  assume  a  different  position 
when  they  laid  back  on  the  table.  Before  the  last  of  three  volume 
acquisitions,  the  volunteer  voided  to  create  an  empty  bladder . 
The  coil  array  was  centered  on  the  prostate.  All  images  of  the 
same  volunteer  were  acquired  with  the  same  MRI  acquisition 
parameters.  In  total,  there  are  12  volumes,  three  for  each  of 
volunteers  S1-S4. 

2)  Imaging  Experiments  on  iMRI  System:  We  acquired 
iMRI  images  under  the  conditions  simulating  the  treatment 
application.  The  volunteer  was  supine,  and  his  legs  were 
supported  at  30° -60°  relative  to  the  horizon  and  separated 
in  a  “V”  with  an  angle  of  60°-90°  between  two  legs.  This  is 
similar  to  the  lithotomy  position  used  in  prostate  therapies,  and 
it  should  provide  access  for  needle  insertion  in  brachytherapy 
or  RF  thermal  ablation.  We  call  this  the  treatment  position. 
Before  experiments,  the  volunteer  voided  their  bladder.  For 
each  volunteer,  all  images  were  obtained  within  a  2-h  session. 
Between  image  acquisitions,  volunteers  moved  to  ensure 
a  different  position.  For  each  of  the  volunteers  SI -S3,  we 
acquired  three  volumes  and  50  iMRI  image  slices  covering 
the  prostate.  They  included  30  transverse,  ten  coronal,  and 
ten  sagittal  image  slices.  We  call  these  images  “ actual ”  iMRI 
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images  to  differentiate  them  from  previous  experiments  using 
“ simulated ”  iMRI  slices. 

D.  Registration  Experiments 

1 )  Registration  Experiments  Using  Simulated  iMRI 
Images:  We  used  12  pairs  of  high-resolution  MR  volumes  to 
perform  registration  experiments.  For  each  volume  pair,  we 
extracted  data  from  one  volume  to  simulate  thick  iMRI  image 
slices;  and  then  we  registered  the  simulated  image  slices  to 
the  other  volume.  We  desire  an  iMRI  slice  image  acquisition 
method  that  gives  robust,  accurate  registrations  and  is  relatively 
insensitive  to  acquisition  parameters.  Hence,  we  performed 
experiments  to  determine  the  dependence  on  slice  orientation 
(transverse,  sagittal,  and  coronal),  on  slice  position  relative  to 
the  prostate  (above,  centered,  and  below),  on  image  noise  from 
fast  imaging  techniques,  and  on  the  inhomogeneous  sensitivity 
response  from  a  belt  coil. 

2)  Registration  Experiments  Using  Actual  iMRI  Image 
Slices:  We  also  performed  two  types  of  SV  registration 
experiments  using  the  actual  iMRI  images.  First,  we  registered 
actual  iMRI  image  slices  with  high-resolution  (1.5-T  system) 
MR  volumes  and  visually  evaluated  results.  For  each  volunteer 
S1-S3,  there  were  three  high-resolution  MR  volumes  and  50 
iMRI  image  slices  giving  150  SV  registration  experiments, 
and  a  total  of  450  experiments.  Second,  we  registered  thick 
slices  simulated  from  the  volume  of  image  data  obtained  on  the 
iMRI  scanner  with  the  corresponding  high-resolution  (1.5-T 
scanner)  MR  volume.  In  this  case,  we  compared  results  to  VV 
registration  obtained  by  registering  the  volume  from  the  iMRI 
system  with  the  high-resolution  volume  (1.5-T  scanner).  We 
investigated  the  effect  of  iMRI  slice  thickness  by  averaging 
1-10  contiguous  image  slices  to  create  a  thick  slice  and 
registering  it  to  the  high-resolution  volume.  The  original  actual 
iMRI  volumes  have  a  slice  thickness  of  1.4  mm  and  in-slice 
dimensions  of  1.3  x  1.3  mm.  We  used  trilinear  interpolation 
to  create  isotropic  actual  iMRI  volumes  with  voxel  size  of 
1.3  x  1.3  x  1.3  mm.  Thus,  thick  slices  simulated  from  actual 
iMRI  volumes  are  1.3  to  13  mm. 

E.  Registration  Evaluation 

1 )  Visual  Inspection:  We  evaluated  registration  experiments 
by  visual  inspection.  We  used  RegViz ,  a  program  created  in 
IDL  in  our  laboratory  with  multiple  visualization  and  analysis 
methods.  First,  we  manually  segmented  prostate  boundaries  in 
image  slices  and  copied  them  to  corresponding  slices.  This  en¬ 
abled  visual  determination  of  the  overlap  of  prostate  boundaries 
over  the  entire  volume.  Second,  color  overlay  displays  were 
used  to  evaluate  overlap  of  structures.  One  image  was  rendered 
in  gray  and  the  other  in  the  “hot-iron”  color  scheme  available 
in  IDL.  To  visualize  potential  differences,  it  was  quite  useful  to 
interactively  change  the  contribution  of  each  image  using  the 
transparency  scale.  Third,  we  used  a  sector  display,  which  di¬ 
vided  the  reference  and  registered  images  into  rectangular  sec¬ 
tors  and  created  an  output  image  by  alternating  sectors  from  the 
two  input  images.  Even  subtle  shifts  of  edges  would  be  clearly 
seen. 

2)  Volume-to-Volume  Registration  Standard:  Our  standard 
evaluation  method  was  to  compare  SV  and  VV  registration. 
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The  W  registration  accuracy  was  previously  evaluated  [8],  For 
volume  pairs  acquired  over  a  short  time  span  from  a  supine  sub¬ 
ject  with  legs  flat  on  the  table,  prostates  were  well  aligned  and 
prostate  centroid  displacements  were  typically  <  1  mm.  The  reg¬ 
istration  accuracy  as  determined  from  displacements  of  pelvic 
bony  landmarks  was  1.6  mm  ±  0.2  mm.  This  error  might  be 
overestimated  because  it  includes  the  uncertainty  of  locating 
the  bony  landmarks.  From  our  success  with  VV  prostate  reg¬ 
istration,  we  decided  that  we  could  obtain  SV  accuracy  by  com¬ 
paring  to  VV  registration  for  those  volume  pairs  having  low  VV 
registration  error. 

To  compare  SV  and  VV  registration  results,  we  defined  a 
rectangular  volume  of  interest  (VOI)  just  covering  the  prostate 
over  which  to  calculate  registration  error.  To  voxels  within  the 
VOI,  we  applied  the  transformations  obtained  by  the  VV  and  by 
SV  registrations.  We  then  calculated  the  3-D  displacements  be¬ 
tween  the  transformed  voxels.  The  mean  voxel  distance  over  the 
VOI  was  used  as  our  metric  of  SV  registration  error.  For  eval¬ 
uation  of  algorithm  robustness,  we  defined  the  SV  registration 
as  being  successful  when  the  mean  3-D  displacement  was  less 
than  2.0  mm. 


IV.  Results 

A.  Experiments  with  Simulated  iMRI  Images  from  the 
7.5-  T  System 

As  described  in  Section  III,  we  obtained  relatively  low-noise 
high-resolution  MR  images  and  simulated  SV  registration  re¬ 
sults.  These  datasets  allowed  us  to  test  effects  of  noise  and  re¬ 
ceive  coil  inhomogeneity  in  a  controlled  fashion.  And,  because 
we  had  substantial  previous  experience  showing  the  accuracy  of 
VV  registration  under  comparable  conditions,  we  could  easily 
determine  SV  error  by  comparing  results  to  VV  registration. 

In  Fig.  7,  the  sector  display  shows  a  simulated  image  slice 
registered  with  a  high-resolution  image  volume.  The  simulated 
image  slice  was  obtained  at  a  transverse  orientation  near  the 
center  of  the  prostate.  The  sector  display  shows  close  align¬ 
ment  at  this  position.  Other  transverse  images  were  also  well 
aligned,  indicating  that  the  registration  was  successful  in  three 
dimensions. 

We  determined  SV  registration  results  for  slices  near  the 
prostate  in  the  three  standard  orthogonal  orientations.  Com¬ 
pared  with  W,  mean  and  standard  deviation  registration  errors 
across  12  volume  pairs  and  60  SV  registration  experiments  were 
0.4  mm  ±  0.2  mm,  0.5  mm  ±  0.2  mm,  and  2.6  mm  ±1.6  mm 
for  transverse,  coronal,  and  sagittal  slices  covering  the  prostate, 
respectively.  Transverse  slices  worked  best  because  they 
contain  many  relatively  rigid  anatomical  structures  (see  Fig.  3). 
We  further  found  that  transverse  slices  centered  on  the  prostate 
produced  better  results  than  those  above  or  below  the  prostate. 
Image  slices  above  included  the  deformable  bladder  that  could 
give  an  inconsistent  structure  from  one  volume  to  the  next. 
Image  slices  below  the  prostate  mainly  contained  muscle  and 
fatty  regions  from  the  hips  that  could  deform,  again  giving 
inconsistent  image  data.  Coronal  slices  worked  next  best. 
Sagittal  slices  gave  the  largest  error  because  they  contained  a 
large  portion  of  the  deformable  bladder  and  rectum. 


Fig.  7.  Sector  display  showing  quality  of  S  V  registration.  Transverse  slices  are 
shown  for  (a)  simulated  iMRI  and  (b)  high-resolution  MRI  images.  In  the  sector 
display  (c),  a  checker  board  pattern  is  created  where  image  sections  from  (a)  and 
(b)  are  alternated.  Square  sections  from  (a)  are  made  brighter  in  order  to  show 
the  boundaries.  As  indicated  by  the  arrows,  the  boundaries  of  bones  and  other 
structures  are  continuous  across  the  sections  indicating  excellent  registration. 
The  prostate  registered  very  well.  Images  are  acquired  from  volunteer  S4. 

Simulation  experiments  showed  SV  registration  to  be  very 
insensitive  to  noise.  We  performed  over  150  registration  exper¬ 
iments  with  noise  added  to  give  SNRs  ranging  from  20  to  5. 
Using  the  slice  configurations  recommended  above  (transverse 
slices  near  the  prostate  center),  we  obtained  100%  successful 
registrations  (an  error  <2.0  mm)  for  SNRs  >10,  a  value  much 
worse  than  the  clinical  SNR  value  of  «25  on  our  iMRI  system. 

Receive  coil  inhomogeneity  also  had  little  effect  on  regis¬ 
tration.  Registration  again  was  100%  successful  for  all  volume 
pairs  under  all  receive  coil  configurations,  even  when  the  coil 
for  the  slice  acquisition  was  displaced  up  to  200  mm  toward  the 
head  from  the  prostate  center,  the  position  of  the  coil  for  the 
volume  acquisition. 

B.  Experiments  with  Actual  iMRI  Images 

Fig.  8  shows  results  for  an  SV  registration  of  actual  iMRI 
image  slices  with  a  high-resolution  MR  volume.  The  contours 
overlap  and  overlay  images  show  that  the  prostate  matches  very 
well.  Other  visual  inspection  techniques  also  demonstrate  ex¬ 
cellent  registration.  Note  that  a  single  iMRI  image  was  used  to 
produce  this  registration  result. 

Fig.  9  shows  SV  registration  error  as  a  function  of  slice  thick¬ 
ness.  As  described  previously,  we  first  registered  each  volume 
from  the  iMRI  scanner  with  the  corresponding  high-resolution 
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Fig.  8.  Images  after  SV  registration  of  actual  iMRI  slices  from  a  0.2-T  open  MR  system.  Image  (a)  is  a  transverse  slice  from  a  high-resolution  MR  volume  (1.5-T 
scanner).  The  prostate  is  segmented  and  magnified  in  image  (b).  Image  (c)  is  the  actual  iMRI  slice  (0.2-T  scanner).  Images  (c)  and  (b)  are  displayed  together  in  an 
overlay  in  image  (d),  and  the  white  rectangular  region  is  magnified  in  image  (e).  The  segmented  prostate  boundary  from  the  high-resolution  MR  image  is  copied 
to  the  actual  iMRI  image  where  it  closely  matches  the  prostate  in  the  actual  iMRI  image  slice  indicating  excellent  registration. 


Fig.  9.  SV  registration  using  images  with  different  slice  thickness.  The  error 
metric  is  the  average  voxel  displacement  between  the  SV  and  VV  registrations. 
Plotted  are  mean  errors  as  well  as  standard  deviation  from  a  rectangular  VOI 
surrounding  the  prostate.  One  typical  datasets  of  high-resolution  MRI  volume 
and  actual  iMRI  slices  of  volunteer  S 1  are  used  for  the  registration  experiments. 
For  each  thickness,  ten  registration  experiments  were  conducted  using  ten 
different  simulated  iMRI  transverse  slices  that  intersected  the  prostate  with 
different  distances.  Thick  iMRI  slices  were  obtained  by  averaging  1-10  iMRI 
image  slices. 

MRI  volume  (1.5-T  scanner)  using  rigid-body  voxel-based  reg¬ 
istration  [8]  and  used  the  result  as  the  gold  standard  for  calcu¬ 
lating  the  SV  error.  Each  thick  slice  image  was  obtained  by  av¬ 
eraging  several  contiguous  slices  from  the  actual  iMRI  volume. 
As  the  slice  thickness  increases  from  1  x  1.3  mm  to  4  x  1.3  mm, 
the  registration  error  decreases,  possibly  because  of  improved 


SNR  and/or  because  of  the  inclusion  of  more  features.  Error  in¬ 
creases  with  thicker  slices,  probably  because  of  the  inconsis¬ 
tency  of  image  features  between  the  thick  slice  and  more  finely 
sampled  volume. 

In  Fig.  10,  we  evaluated  SV  registration  for  thick  slices  at 
different  orientations.  The  evaluation  method  was  the  same  as 
that  used  in  Fig.  9,  and  the  slices  were  5  mm  thick  and  inter¬ 
sected  the  volume  near  the  prostate  center.  Results  were  con¬ 
sistent  with  those  from  the  previous  simulation  experiments. 
Transverse  slices  worked  best  with  an  average  VOI  displace¬ 
ment  of  only  1.1  mm  ±  0.7  mm  and  a  success  rate  of  100%. 
The  coronal  images  gave  a  reasonable  average  error,  but  the  suc¬ 
cess  rate  dropped  to  86%.  The  sagittal  orientation  gave  the  worst 
result. 

Needle  artifacts  had  little  effect  on  the  S  V  registration.  In  each 
of  30  the  experiments,  we  registered  a  high-resolution  volume 
with  an  actual  iMRI  image  slice  containing  or  not  containing 
a  simulated  needle  artifact.  Visual  inspection,  the  correlation 
coefficient,  and  mutual  information  values  of  registered  images 
showed  little  effect  of  the  needle  artifact.  The  success  rate  was 
100%  in  both  cases. 

C.  Algorithmic  Robustness  and  Implementation 

The  registration  algorithm  was  quite  robust  for  transverse 
slices  covering  the  prostate.  Using  simulated  iMRI  slices  from 
high-resolution  MRI  volume  pairs  of  four  volunteers,  the  algo¬ 
rithm  never  failed  for  any  transverse  slice  covering  the  prostate. 
In  addition,  the  final  registration  result  was  insensitive  to  initial 
guesses  within  a  very  large  range,  [—60,  +60]  mm  for  transla¬ 
tions  and  [—20,  +20]  degrees  for  rotations.  With  the  restarting 
algorithm,  we  even  successfully  registered  slices  as  much  as 
80  mm  from  the  optimum.  This  working  range  should  be  quite 
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Fig.  10.  SV  registration  error  and  robustness  for  iMRI  images  in  the  three 
standard  orientations.  In  (a),  registration  error  relative  to  VV  registration  is 
plotted  as  a  function  of  image  slice  orientation.  In  (b),  success  rate  is  also  plotted 
as  a  function  of  orientation  where  registration  is  successful  when  the  error  is 
<2.0  mm.  For  volunteer  S2,  one  high-resolution  volume  and  one  volume  from 
the  iMRI  scanner  were  used  in  these  experiments.  Data  were  extracted  from  the 
iMRI  volume  to  simulate  iMRI  slices  with  a  thickness  of  about  5  mm.  Fifteen 
transverse,  coronal,  and  sagittal  slices  from  the  prostate  center  were  used  for  SV 
registration,  respectively. 

sufficient  for  clinical  applications  where  we  can  ensure  good 
starting  values.  Using  the  pelvic  bones  as  markers  and  device 
localization  methods  [29],  we  should  be  able  to  position  the 
prostate  within  about  ±20  mm  in  the  imaging  field.  In  addition, 
the  patient  normally  lies  supine  in  the  MR  bed  with  very  little 
rotation  (<  ±5°). 

Using  CC  and  MI  at  different  resolutions  was  an  important 
feature  that  increased  robustness.  MI  registrations  at  low  reso¬ 
lution  sometimes  gave  false  maxima  [Fig.  1(a)  and  (c)],  and  only 
60%  success  was  achieved  when  MI  was  used  at  all  resolutions. 
The  interpolation  artifacts  at  low  resolutions  often  caused  fail¬ 
ures  and  required  more  restarts  [16].  CC  performed  well  and 
gave  fewer  local  maxima  at  the  lower  resolutions  [Fig.  1(b)  and 
(d)],  but  MI  was  more  accurate  than  CC  at  the  highest  resolution 
due  to  the  sharper  peak  of  the  MI  surface  [Fig.  2(a)  and  (c)]  [8], 
Our  registration  algorithm  thus  combined  advantages  from  the 
two  similarity  measures. 


The  multiresolution  approach  improved  algorithmic  robust¬ 
ness  and  speed.  When  we  used  only  MI  at  full  resolution, 
registration  was  70%  successful  compared  to  the  100%  of  the 
full  algorithm.  This  failure  of  MI  was  also  reported  by  others 
[13],  [17].  The  multiresolution  approach  enabled  the  program 
to  quickly  approach  the  final  value  because  of  the  reduced 
number  of  calculations  at  low  resolutions.  For  a  typical  image 
pair,  iterations  at  1/4  resolution  were  approximately  4  and  25 
times  faster  than  at  1/2  and  full  resolution,  respectively. 

Restarting  was  important  for  image  pairs  with  large  trans¬ 
lations  and/or  rotations  from  the  optimum.  In  our  experience 
with  over  800  SV  registration  experiments,  restarting  occurred 
in  about  5%  of  them.  For  an  example  pair  with  an  80-mm  dis¬ 
placement,  the  number  of  restarts  was  3, 1,  and  0  at  1/4, 1/2,  and 
full  resolutions,  respectively.  Without  restarting,  we  found  that 
registrations  sometimes  failed  in  cases  of  volumes  with  a  large 
mismatch  of  54  mm  and  high  noise.  The  algorithm  was  insensi¬ 
tive  to  the  CC  threshold  for  restarting.  When  we  decreased  the 
threshold  from  0.8  to  0.5  with  an  interval  of  0.05,  we  found  little 
change  in  the  number  of  restarts  and  no  change  in  final  registra¬ 
tions.  We  set  the  threshold  at  0.5  to  avoid  only  the  most  obvious 
local  maxima. 

We  now  describe  some  aspects  of  the  implementation.  The 
time  for  an  SV  registration  was  typically  about  15  s  on  a  Pen¬ 
tium  IV  1.8-GHz  CPU  with  1  GB  of  memory.  The  algorithm 
was  written  in  IDL  and  could  probably  be  made  much  faster  in 
a  lower  level  language  such  as  C.  A  call  to  the  Simplex  optimiza¬ 
tion  typically  resulted  in  50  to  105  similarity  evaluations  before 
the  tolerance  value  (0.001)  was  reached.  The  simplex  optimiza¬ 
tion  method  worked  about  1. 5-2.0  times  faster  than  the  Powell 
method  in  our  implementation.  We  used  the  Simplex  method  for 
our  experiments  in  this  study. 

V.  Discussion  and  Conclusion 

Despite  complications  such  as  image  noise,  receive  coil  in¬ 
homogeneity,  a  limited  number  of  voxels,  and  needle  artifacts, 
SV  voxel-based  registration  can  be  quite  robust  and  accurate. 
For  transverse  slices  covering  the  prostate,  registration  results 
agreed  very  favorably  with  VV  results.  Below,  we  further  dis¬ 
cuss  the  algorithm  and  its  practicality. 

A.  Mutual  Information  at  Low  Resolution 

There  are  probably  several  reasons  why  mutual  information 
does  not  work  well  at  low  resolution.  First,  the  similarity  curve  is 
noisy  with  periodic  oscillations  from  the  so-called  interpolation 
artifact  [8],  [16]  that  is  accentuated  at  reduced  resolutions  [30]. 
As  a  result,  there  are  many  local  maxima  in  Fig.  1(a)  and  (c)  that 
can  trap  the  optimization;  and  a  similar  result  was  reported  for 
brain  registration  [13].  In  additional  experiments,  we  decreased 
the  number  of  bins  for  both  images  to  256,  128,  64,  and  32  and 
plotted  mutual  information  values  as  a  function  of  translation. 
With  a  larger  number  of  bins,  we  got  no  discernable  effect  of  bin 
size.  When  the  number  of  bins  was  reduced  to  32,  the  MI  surface 
was  degraded.  Others  showed  that  Gaussian  blurring  of  images 
before  registration  did  not  improve  performance  at  low  resolu¬ 
tions  and  that  there  was  little  difference  between  standard  and 
normalized  mutual  information  [40].  Second,  when  images  are 
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of  low  resolution  and  there  is  only  a  small  region  of  overlap,  the 
mutual  information  function  can  even  contain  incorrect  global 
maxima  [30]  as  found  in  Fig.  1(a).  This  false  result  was  obtained 
at  very  large  displacements  where  the  SV  overlap  was  reduced. 
This  occurs  because  MI  is  not  only  a  function  of  how  well  the 
images  match  in  the  overlap,  but  also  by  how  much  information 
is  provided  by  the  two  images  in  the  overlap  [31],  [32],  [35].  As 
shown  above,  using  both  mutual  information  and  correlation  co¬ 
efficient,  at  different  resolutions,  was  an  important  feature  that 
increased  robustness. 

B.  Accuracy  Consideration 

Essentially,  we  found  that  SV  is  of  similar  accuracy  to  VV 
registration,  with  an  average  voxel  displacement  difference  of 
only  0.4  mm  in  the  prostate  for  the  simulated  images  and  about 
1  mm  for  actual  iMRI  image  data.  Hence,  the  accuracy  of  the 
best  SV  method  is  essentially  that  previously  reported  for  VV 
registration  [8]. 

We  recommend  that  image  data  are  obtained  under  compa¬ 
rable  conditions  by  keeping  a  similar  posture  and  by  taking 
clinical  measures  to  reduce  rectal  and  bladder  filling.  We  see 
no  reason  to  suspect  that  SV  registration  will  be  inaccurate 
when  such  conditions  are  met.  When  images  were  acquired 
under  much  different  conditions,  such  as  legs  flat  and  legs 
raised,  rigid-body  registration  could  result  in  prostate  centroid 
errors  as  much  as  3.4  mm.  Another  effect  may  be  the  tissue 
deformation  from  insertion  of  the  RF  needle.  From  our  previous 
experience  observing  in  vivo  needle  insertion  in  both  animal 
models  and  clinical  trials  with  real-time  MRI,  the  amount  of 
tissue  deformation  that  occurs  with  insertion  of  a  sharp  bevel 
tip  needle  is  minimal  and  transient  in  tissues  with  normal 
interstitial  pressure.  In  certain  lesions,  such  as  cysts  or  necrotic 
tumor,  persistent  deformation  is  possible;  however,  we  can 
see  such  deformations  in  the  live-time  interventional  MRI 
images  and  very  probably  mentally  correct  the  registered, 
fused  images.  We  previously  reported  a  warping  registration 
method  [38],  [39]  that  can  correct  deformations  at  the  expense 
of  additional  complexity,  time,  and  possibly  robustness. 

The  automatic  SV  registration  provides  sufficient  accuracy 
for  many  potential  iMRI  applications.  As  compared  to  a  typical 
SPECT  and/or  iMRI  slice  thickness  of  >3.0  mm,  SV  registra¬ 
tion  is  quite  accurate.  MR  spectroscopy  also  is  done  at  limited 
resolution.  If  one  were  to  use  functional  or  high-resolution  MR 
images  directly  for  targeting,  the  requirements  for  registration 
accuracy  would  be  great.  However,  fused  image  data  will  not  be 
used  blindly.  Rather,  these  visualizations  will  be  used  as  a  guide. 
Physicians  will  always  use  the  live-time  iMRI  images  for  needle 
guidance.  With  proper  visualization  tools,  they  should  be  able  to 
mentally  account  for  any  small  registration  errors.  In  addition, 
very  often  there  is  image  evidence  of  cancer  in  MR  prostate  im¬ 
ages  that  can  perhaps  be  identified  with  the  aid  of  functional 
images.  Such  MR-visible  lesions  can  then  become  the  markers 
for  tumor  targeting. 

C.  Practicality  and  Application 

The  registration  experiments  presented  here  provided 
fairly  comprehensive  tests  for  the  potential  application  in 
iMRI-guided  RF  thermal  ablation  of  the  prostate.  Simulation 
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provided  an  efficient  way  to  extensively  evaluate  registration 
performance.  The  algorithm  was  extremely  robust  to  noise 
levels,  far  beyond  those  encountered  in  clinical  iMRI  appli¬ 
cations.  Similarly,  the  inhomogeneity  seen  with  a  belt  coil 
was  not  problematic  for  transverse  images,  probably  due  to 
coil  inhomogeneity  simply  scaling  the  grayscale  values,  an 
operation  that  should  not  affect  MI  or  CC  similarity  measures. 
Needle  artifacts  had  little  effect,  probably  because  they  occupy 
relatively  few  voxels.  The  actual  iMRI  images  acquired  under 
more  realistic  conditions  further  tested  practicality.  Images 
from  the  iMRI  system  contained  more  noise  and  had  less 
contrast  than  those  from  the  1.5-T  scanner.  Registration  quality 
was  comparable  to  that  of  simulation  experiments.  Registration 
time  can  probably  be  improved  considerably  using  optimized  C 
code  rather  than  IDL.  If  registration  is  done  in  the  background 
in  a  seamless  way,  the  time  for  registration  is  probably  quite 
acceptable.  Although  we  normally  used  T2-weighted  image 
pairs,  the  registration  worked  well  for  pairs  of  Tl-weighted  and 
T2- weigh  ted  images. 

We  conclude  that  the  automatic  SV  registration  algorithm  is 
quite  robust  for  transverse  image  slices  covering  the  prostate 
and  that  the  registration  provides  sufficient  accuracy  to  aid 
image-guided  therapy.  From  previous  reports  of  MR-PET  or 
MR-SPECT  registration  accuracy  [6],  [7],  it  appears  feasible  to 
combine  functional  images  to  aid  iMRI-guided  procedures.  We 
are  beginning  to  explore  this  application  in  animal  experiments. 
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Abstract  --We  are  investigating  interventional  MRI  (iMRI) 
guided  radiofrequency  thermal  ablation  for  the  minimally 
invasive  treatment  of  the  prostate  cancer.  Nuclear  medicine  can 
detect  and  localize  tumor  in  the  prostate  not  reliably  seen  in 
MRI.  We  intend  to  combine  the  advantages  of  functional  images 
such  as  nuclear  medicine  SPECT  with  iMRI-guided  treatments. 
Our  concept  is  to  first  register  the  low-resolution  SPECT  with  a 
high  resolution  MRI  volume.  Then  by  registering  the  high- 
resolution  MR  image  with  live-time  iMRI  acquisitions,  we  can, 
in  turn,  map  the  functional  data  and  high-resolution  anatomic 
information  to  live-time  iMRI  images  for  improved  tumor 
targeting.  For  the  first  step,  we  used  a  three  dimensional  mutual 
information  registration  method.  For  the  latter,  we  developed  a 
robust  slice  to  volume  (SV)  registration  algorithm  with  special 
features.  The  concept  was  tested  using  image  data  from  three 
patients  and  three  volunteers.  The  SV  registration  accuracy  was 
0.4  mm  ±  0.2  mm  as  compared  to  our  volume-to-volume 
registration  that  was  previously  shown  to  be  quite  accurate  for 
these  image  pairs.  With  our  image  registration  and  fusion 
software,  simulation  experiments  show  that  it  is  quite  feasible  to 
incorporate  SPECT  and  high  resolution  MRI  into  the  iMRI- 
guided  minimally  invasive  treatment  procedures. 


i.  Introduction 

e  use  an  interventional  magnetic  resonance  imaging 
(iMRI)  system  to  guide  minimally  invasive  treatments, 
including  the  radiofrequency  (RF)  thermal  ablation  of 
abdominal  cancers  [l]-[3].  The  iMRI  system  consists  of  a 
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0.2  T,  clinical  C-arm  open  MRI  scanner,  an  in-room  RF- 
shielded  liquid  crystal  monitor,  an  MR  compatible  mouse,  a 
foot  pedal,  and  a  RF  ablation  device.  We  are  currently 
investigating  the  extension  of  these  techniques  to  the 
treatment  of  prostate  cancer.  Since  MRI  does  not  reliably 
show  prostate  tumors,  we  intend  to  incorporate  nuclear 
medicine  SPECT  or  MR  spectroscopy  images  with  higher 
sensitivity  for  detecting  and  localizing  prostate  tumors  [4] [5]. 

To  incorporate  image  data  from  other  sources  in  a  live-time 
iMRI  procedure,  we  intend  to  register  two-dimensional  (2D) 
slice  images  quickly  acquired  on  the  iMRI  scanner  in  live¬ 
time  with  a  previously  acquired  volume  of  image  data.  Then, 
to  incorporate  an  image  volume  from  another  modality,  it  can 
be  registered  with  the  full  MR  volume.  Thus,  to  incorporate 
SPECT  in  an  iMRI  procedure,  we  will  first  register  the 
SPECT  image  volume  with  a  high  resolution  MR  volume; 
then,  when  we  register  iMRI  slice  images  to  the  high 
resolution  MR  volume,  we  can  also  map  them  to  the  SPECT 
functional  image  data.  If  this  procedure  is  successful,  then  a 
variety  of  potential  visualization  tools  can  help  the  physician 
appropriately  localize  and  apply  treatments.  The  live-time 
iMRI  images  will  be  used  for  guidance,  and  very  probably 
any  small  misregistration  errors  can  be  mentally  corrected  by 
the  physician.  To  simplify  and  possibly  improve  the  slice-to- 
volume  (SV)  registration  step,  we  intend  to  always  use  MR 
images  acquired  with  similar  pulse  sequences. 

The  application  of  S  V  registration  methods  to  iMRI-guided 
treatment  of  prostate  cancer  raises  several  challenges.  First, 
iMRI  images  often  have  lower  signal  to  noise  ratio  (SNR) 
than  diagnostic  MR  images  because  of  the  emphasis  on  fast 
imaging  and  because  of  the  typically  lower  field  strength  of 
open  iMRI  magnets.  Second,  a  single  slice,  or  a  few  slices, 
provides  many  fewer  structures  than  an  entire  volume  for 
voxel  based  matching.  Third,  the  prostate  can  move  relative 
to  the  pelvic  bones  due  to  changes  in  rectal  and  bladder 
filling  [6]  or  changes  in  patient  posture  for  treatment  [7].  That 
is,  alignment  of  the  pelvic  bones,  prominent  anatomical 
features  in  MR  gray-scale  images,  does  not  necessarily  ensure 
that  the  prostate  is  aligned.  Fourth,  the  normal  prostate  is  a 
small  organ;  when  healthy,  it  measures  only  ~  3.8  cm  in  its 
widest  dimension  [8].  The  small  prostate  is  located  below  the 
much  larger  bladder  that  can  change  its  shape  and  size  during 
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imaging.  Finally,  times  for  registration  and  algorithm 
robustness  are  of  particular  concern  for  this  application. 

Previous  success  with  registering  one  MR  prostate  volume 
to  another  [7]  encourages  us  to  pursue  this  plan.  We  call  this 
volume-to-volume  registration,  or  VV.  We  used  a  rigid  body, 
mutual  information  registration  method  with  some  features  to 
improve  robustness  [7].  We  carefully  evaluated  registration 
quality  using  a  variety  of  methods.  For  volume  pairs  acquired 
over  a  short  time  span  from  a  supine  subject  with  legs  flat  on 
the  table,  registration  accuracy  of  both  prostate  centroids 
(typically  <  1  mm)  and  bony  landmarks  (average  1 .6  mm)  was 
on  the  order  of  a  voxel  (~  1 .4  mm).  For  volumes  acquired 
under  very  different  conditions,  e.g.,  legs  flat  and  legs  raised 
into  the  treatment  position,  or  with  and  without  bladder  or 
rectal  filling,  we  obtained  somewhat  larger  prostate  centroids 
registration  errors  of  about  3.0  mm.  From  our  results  with  VV 
prostate  registration,  we  decided  that  we  could  assess  SV 
accuracy  by  comparing  results  to  VV  registration  for  those 
volume  pairs  having  low  VV  registration  error. 

In  the  next  sections,  we  will  report  algorithms  and  results 
for  the  slice-to-volume  registration  between  an  iMRI  thick 
slice  and  a  high  resolution  MRI  volume,  the  three 
dimensional  registration  of  SPECT  and  high  resolution  MRI 
volumes,  and  the  fusion  of  the  three  modalities  for  potential 
applications  in  iMRI-guided  thermal  ablation  of  the  prostate. 


II.  Registration  Algorithms 

A.  Similarity  Measures 

For  slice  to  volume  registration  of  iMRI  image  slice  and 
high  resolution  MRI  volume,  we  used  two  similarity 
measures,  mutual  information  and  correlation  coefficient,  in 
our  registration.  Suppose  one  image  R  is  the  reference ,  and 
the  other  F  is  floating.  Their  mutual  information  MI(R,F)  is 
given  below  [9]. 

MI(R,  F)  n  □  p„  (r,  /)  log 

rj  Pn(r)  CM/) 

The  joint  probability  pRF(r,f)  and  the  marginal  probabilities 
pR(r)  of  the  reference  image  and  pF(f)  of  the  floating  image, 

can  be  estimated  from  the  normalized  joint  intensity 
histogram. 

The  correlation  coefficient  (CC)  is  the  measure  of  the 
interdependence  of  two  random  variables  that  ranges  in  value 
from  -1  to  +1,  indicating  perfect  negative  correlation  at  -1, 
absence  of  correlation  at  zero,  and  perfect  positive  correlation 
at  +1.  For  the  reference  and  floating  images,  R  and  F,  their 
correlation  coefficient  CC(R,F)  is  given  below  The 
correlation  coefficient  CC(R,F)  is  given  below  [10]. 

□W)  M)(F(0  □  F) 
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Here  r^  f  denote  the  average  intensities  of  the  reference  and 

floating  volumes  and  the  summation  includes  all  N  voxels 
within  the  overlap  of  both  volumes. 

B.  Registration  of  iMRI  Slice  and  High  Resolution  MRI 

Volume 

We  used  a  registration  algorithm  similar  to  the  one  as 
previously  reported  by  us  [  1 1  ]  [  1 2] .  We  used  multi-resolution 
approach  and  perform  registration  from  low  to  high 
resolution.  At  1/4  resolution ,  we  resampled  images  so  as  to 
give  1/4  number  of  the  voxels  along  each  linear  dimension.  At 
full  resolution ,  we  used  the  full  number  of  voxels.  We  use 
correlation  coefficient  at  the  two  lower  resolutions  because  it 
gives  fewer  local  maximums  and  because  it  can  be  calculated 
faster  than  MI.  We  use  MI  at  full  resolution  because  the 
peaked  similarity  function  gives  a  more  precise  solution  than 
CC  [7].  To  avoid  local  maximums,  we  include  a  restarting 
feature  where  registration  is  restarted  with  randomly 
perturbed  parameters  obtained  from  a  uniform  distribution 
about  the  initial  transformation  values  at  the  current 
resolution  being  used.  The  algorithm  restarts  until  the 
absolute  CC  is  above  an  experimentally  determined  threshold 
or  the  maximum  number  of  restarts  is  reached.  Absolute  CC 
is  used  rather  than  MI  because  it  has  a  well-defined  range 
between  0  and  1  and  because  it  provides  an  independent 
check  of  the  MI  result  at  the  highest  resolution. 

We  record  all  important  results  following  an  optimization 
cycle  including  the  CC  and/or  MI  values,  the  number  of 
restarts,  and  the  transformation  parameters.  At  the  end  of 
processing  at  a  lower  resolution,  we  always  select  the 
transformation  parameters  having  the  maximum  CC  value. 
We  then  scale  the  translation  parameters  appropriately  and 
assign  the  new  parameters  to  be  initial  values  at  the  next 
higher  resolution.  At  the  highest  resolution,  MI  instead  of  CC 
is  the  similarity  measure,  and  we  select  the  final 
transformation  parameters  to  be  those  with  the  maximum  MI 
value. 

Typical  parameter  values  are  now  described.  We  use  an 
initial  guess  assuming  an  identity  transformation,  i.e.,  all 
initial  translation  and  rotation  parameters  are  zero,  because 
the  patient  is  normally  oriented  approximately  the  same  way 
from  one  scan  to  the  next.  The  algorithm  restarts  until  the 
absolute  CC  is  above  a  threshold  of  0.5  or  maximum  numbers 
of  restarts  are  reached  (10,  5,  and  3,  from  low  to  high 
resolution,  respectively). 

C.  Registration  of  SPECT  and  High  Resolution  MRI 

Volume 

The  mutual  information  algorithm  was  used  to  register 
MRI  and  SPECT  volume  images  because  of  its  ability  to  align 
multi-modality  images  [13][14],  Registration  of  SPECT  and 
MR  images  is  challenging  because  the  two  image  types  have 
different  spatial  resolutions  and  image  features.  The 
radiotracer  used  for  SPECT  imaging  was  ProstaScintfj 
(Cytogen  Corporation,  Princeton,  NJ),  a  monoclonal  antibody 
that  binds  to  prostate-specific  membrane  antigen  (PSMA). 
Before  registration,  both  SPECT  and  MRI  volumes  were 
resized  using  trilinear  interpolation  to  create  volumes  matrix 
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of  128x128x128  with  3  mm  isotropic  voxels,  a  voxel  size 
between  that  of  the  two  scans.  The  standard  parameter  set  for 
automatic  registration  included:  256  intensity  levels  for  each 
volume,  the  entire  2D  joint  histogram,  the  full  field  of  view  of 
128x128x128  voxels  for  both  volumes,  and  no  masking  or 
cropping  of  either  volume.  Phantom  data  were  preprocessed 
in  a  similar  fashion. 


in.  Experimental  Methods 

A.  High  Resolution  MR  Image  Acquisitions 

High-resolution  MRI  volumes  were  acquired  using  a  1.5  T 
Siemens  MRI  system  (Magnetom  Symphony,  Siemens 
Medical  Systems,  Erlangen,  Germany).  An  8-element  phased 
array  body  coil  was  used  to  ensure  coverage  of  the  prostate 
with  a  uniform  sensitivity.  Typically  two  anterior  and  two 
posterior  elements  were  enabled  for  signal  acquisition.  We 
used  two  different  MR  sequences. 

First,  we  used  a  3D  rapid  gradient  echo  sequence  (PSIF) 
designed  to  acquire  the  spin-echo  component  of  the  steady 
state  response,  rather  than  the  free  induction  decay.  The  spin 
echo  component  forms  immediately  prior  to  the  RF  pulse;  it  is 
shifted  toward  the  prior  RF  pulse  through  appropriate 
gradient  waveform  design.  The  sequence  with  9.4/5.0/60 
(TR/TE/flip)  yields  160x256x128  voxels  over  a 
219x350xl92-mm  rectangular  FOV  and  1.4xl.4xl.5-mm 
voxels  oriented  to  give  the  highest  resolution  for  transverse 
slices.  There  is  over  sampling  at  31%  in  the  slice  direction  to 
reduce  aliasing  artifacts.  The  acquisition  time  is  4  min  and  15 
sec.  This  sequence  gave  excellent  image  contrast  for  the 
prostate  and  its  surroundings.  It  was  used  to  acquire  volumes 
for  volunteers  SI -S3. 

Second,  we  used  a  3D  RF  spoiled  gradient  echo  steady  state 
pulse  sequence  (FLASH)  with  TR/TE/flip  parameters  of 
12/5.0/60  which  give  256x256x  128  voxels  over  a 
330  x  330  x  256-mm  field  of  view  (FOV)  to  yield 
1.3  x  1.3  x  2.0-mm  voxels  oriented  to  give  the  highest 
resolution  for  transverse  slices.  The  acquisition  time  is  5  min 
and  38  sec.  This  sequence  is  good  for  pelvic  imaging  but  is 
not  ideal  for  the  prostate.  It  was  used  to  acquire  volumes  for 
patients  S4-S8. 

When  acquiring  high-resolution  MR  volumes,  volunteers 
laid  supine  in  a  manner  similar  to  the  diagnostic  position  in 
routine  MR  scanning.  Between  volume  acquisitions, 
volunteers  got  up  from  the  MR  table,  stretched,  and  walked 
around  to  ensure  that  they  would  assume  a  different  position 
when  they  laid  back  on  the  table.  The  coil  array  was  centered 
on  the  prostate.  We  acquired  three  volumes  from  each  of  the 
volunteers  S1-S3.  For  patients  S4-S8,  we  acquired  nine  MRI 
volumes  and  each  patient  with  at  least  one  volume. 

B.  Interventional  MRI  Image  Acquisitions  and  Simulation 

We  acquired  iMRI  images  using  a  clinical  0.2  T  C-arm 
open  MR  scanner  (Siemens  Open  Symphony,  Erlangen, 


Germany)  modified  for  interventional  MRI  procedures  and  in 
this  paper  referred  to  as  the  iMRI  system.  We  used  a  two- 
dimensional  PSIF  with  15.2/7.4/45  (TR/TE/FA)  for  image 
slice  acquisitions.  The  iMRI  slices  were  128x128  with  in¬ 
plane  pixel  size  of  2.8x2.8  mm  and  with  effective  slice 
thickness  of  5  mm. 

We  acquired  iMRI  images  under  the  conditions  simulating 
the  treatment  application.  The  volunteer  was  supine,  and  his 
legs  were  supported  at  30  Q60D  relative  to  the  horizon  and 
separated  in  a  “V”  with  an  angle  of  60Q90Dbetween  two  legs. 
This  is  similar  to  the  lithotomy  position  used  in  prostate 
therapies,  and  it  should  provide  access  for  needle  insertion  in 
brachytherapy  or  RF  thermal  ablation.  We  call  this  the 
treatment  position.  For  each  of  the  volunteers  SI -S3,  we 
acquired  30  iMRI  image  slices  covering  the  prostate.  They 
included  10  transverse,  10  coronal,  and  10  sagittal  image 
slices.  We  call  these  images  “actual”  iMRI  images  to 
differentiate  them  from  “simulated”  images  as  described  in 
the  next  paragraph. 

To  test  a  variety  of  clinical  conditions,  we  used  high- 
resolution  MRI  volumes  to  simulate  iMRI  images  by  creating 
thick  slices  and  adding  noise  and  receive  coil  inhomogeneity 
[15].  Clinically,  we  typically  use  an  iMRI  slice  thickness  of 
4.0  -  6.0  mm.  We  averaged  3  slices  1.4  mm  thick  to  create  a 
4.2  mm  thick  slice.  We  added  noise  to  the  simulated  iMRI 
image. 

C.  SPECT  Image  Acquisition 

The  study  included  five  patients  (S4-S8)  with  either  high 
Gleason  scores  (>5)  from  biopsy  or  rising  PSA  level  (>10 
mcg/L,  prostate  specific  antigen)  or  palpation  staging  beyond 
stage  Tl.  After  patient  eligibility  was  established,  patients 
gave  informed  consent.  The  Institutional  Review  Board  of  the 
University  Hospitals  of  Cleveland  approved  the  imaging 
protocol. 

Approximately  four  days  after  injecting  5  mCi 
ProstaScintD,  the  abdominal  and  pelvic  regions  were  scanned 
using  a  two-head  Siemens  E.CAM+  camera  (Siemens  Medical 
System,  Inc.,  Hoffman  Estates,  Illinois,  USA).  ProstaScintD 
is  an  [In-lll]-labeled  monoclonal  antibody  capromab 
penditide  (H1In  MoAb  7E11.C5)  used  for  imaging  prostate 
cancer.  The  evening  before  scanning,  patients  performed  a 
bowel  prep  with  Fleet  R  Prep  Kit  #3  (Fleet  Pharmaceuticals, 
Lynchburg,  VA).  Images  were  acquired  with  a  medium 
energy  collimator  and  15%  energy  window.  The  acquisition 
parameters  included  a  step-and-shoot  motion,  a 
128  x  128  pixel  matrix  for  each  projection,  an  imaging  time 
of  25  sec  per  stop,  and  a  total  of  120  stops  over  a  full  360D 
rotation.  The  field  of  view  of  was  53.3  x  38.7  cm2.  The 
Ordered  Subsets  Expectation  Maximization  (OSEM) 
algorithm  was  used  for  image  reconstruction  [16].  SPECT 
images  were  comprised  of  4.795  x  4.795  x  4.795-mm 
isotropic  voxels.  Each  patient  had  one  SPECT  scan  of  the 
pelvis. 

To  analyze  and  validate  registration  of  high  resolution  MRI 
and  SPECT  under  a  controlled  situation,  an  acrylic  phantom 
of  the  pelvis  and  lower  abdomen  was  used.  Spheres  of 
proportional  size  representing  portions  of  the  bladder, 
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acetabula,  rectum,  and  the  prostate  gland  were  placed  in 
appropriate  positions  in  the  torso  phantom.  The  spheres  of 
acetabulum  were  filled  with  potassium  phosphate.  Other 
spheres  were  filled  with  water.  The  torso  phantom  was  filled 
with  a  small  amount  of  copper  sulfate  dissolved  in  deionized 
water.  The  SPECT  scan  was  conducted  after  injecting  all 
spheres  with  [In-lll]-DTPA  at  relative  concentrations 
comparable  to  those  detected  in  human  scans.  The  water  in 
the  torso  was  given  a  background  activity  of  1  CCi/ml  such  as 
to  mimic  the  background  in  human  SPECT  scans. 

D.  Registration  Experiments 

We  used  9  pairs  of  high-resolution  MR  volumes  of 
volunteers  S1-S3  to  perform  SV  registration  experiments.  For 
each  volume  pair,  we  extracted  data  from  one  volume  to 
simulate  thick  iMRI  image  slices;  and  then  we  registered  the 
simulated  image  slices  to  the  other  volume.  We  desire  an 
iMRI  slice  image  acquisition  method  that  gives  robust, 
accurate  registrations  and  is  relatively  insensitive  to 
acquisition  parameters.  Hence,  we  performed  experiments  to 
determine  the  dependence  on  slice  orientation  (transverse, 
sagittal  and  coronal),  on  slice  position  relative  to  the  prostate 
(above,  centered,  and  below)  and  on  image  noise  from  fast 
imaging  techniques. 

We  also  performed  SV  registration  experiments  using  the 
actual  iMRI  images  from  volunteers  S1-S3.  We  registered 
actual  iMRI  image  slices  with  high-resolution  (1.5  T  system) 
MR  volumes  of  the  same  volunteer  and  visually  evaluated 
results.  For  each  volunteer,  there  were  three  high-resolution 
MR  volumes  and  30  iMRI  image  slices  giving  90  SV 
registration  experiments,  and  a  total  of  270  experiments. 

A  number  of  technical  issues  were  examined  for  MI 
registration  of  MRI  and  ProstaScintD  SPECT  prostate 
images.  Firstly,  MRI  acquisition,  by  varying  the  MR  imaging 
pulse  sequence,  various  structures  can  be  emphasized  or 
suppressed.  Several  different  acquisition  sequences  were 
tested  and  its  effect  on  registration  accuracy  and  robustness 
was  determined.  Secondly,  because  of  the  different  dynamic 
ranges  between  MR  and  SPECT  images,  intensity  scaling  was 
studied  for  its  effect  on  registration.  This  is  prompted  by  a 
recent  study  showing  that  scaling  images  to  16  gray  levels 
gives  better  results  than  256  gray  levels  when  registering 
muscle  fiber  images  [17].  Thirdly,  because  of  the  sparseness 
in  the  histogram,  the  use  of  a  portion  or  a  section  rather  than 
the  full  joint  histogram  was  evaluated.  This  effectively 
restricted  the  registration  to  particular  intensity  ranges. 
Fourthly,  the  multi-resolution  approach  was  examined  for  its 
ability  to  expedite  the  automated  search  algorithm.  Fifthly, 
the  use  of  spatial  masking  was  investigated  to  see  whether  it 
facilitates  the  registration  of  partially  overlapping  volumes.  In 
all  cases,  registration  experiments  were  performed  with  and 
without  these  modifications  to  determine  their  effect  on  the 
success  of  registration.  Success  was  determined  by  comparing 
the  results  of  these  experiments  to  those  of  manual 
registration  of  the  same  images  as  described  in  the  next 
section.  Experiments  with  these  parameters  should  provide 
insight  into  improving  registration  of  MR  and  SPECT 


prostate  images.  We  performed  registration  experiments  using 
the  SPECT  and  MRI  image  volumes  from  patients  S4-S8. 

E.  Registration  Evaluations 

We  evaluated  registration  experiments  by  visual  inspection. 
We  used  RegViz ,  a  program  created  in  IDL  (Interactive  Data 
Language,  Research  System  Inc.,  CO)  in  our  laboratory  with 
multiple  visualization  and  analysis  methods.  First,  we 
manually  segmented  prostate  boundaries  in  image  slices  and 
copied  them  to  corresponding  slices.  This  enabled  visual 
determination  of  the  overlap  of  prostate  boundaries  over  the 
entire  volume.  Second,  color  overlay  displays  were  used  to 
evaluate  overlap  of  structures.  One  image  was  rendered  in 
gray  and  the  other  in  the  “hot  -iron”  color  scheme.  To 
visualize  potential  differences,  it  was  quite  useful  to 
interactively  change  the  contribution  of  each  image  using  the 
transparency  scale.  Third,  we  used  a  sector  display,  which 
divided  the  reference  and  registered  images  into  rectangular 
sectors  and  created  an  output  image  by  alternating  sectors 
from  the  two  input  images.  Even  subtle  shifts  of  edges  would 
be  clearly  seen. 

Our  standard  evaluation  method  for  SV  registration  was  to 
compare  SV  and  W  registration.  Since  this  relies  on  VV 
registration  accuracy,  we  now  review  our  previous  results  [7]. 
For  volume  pairs  acquired  over  a  short  time  span  from  a 
supine  subject  with  legs  flat  on  the  table,  prostates  were  well 
aligned  and  prostate  centroid  displacements  were  typically 
<  1  mm.  The  registration  accuracy  as  determined  from 
displacements  of  pelvic  bony  landmarks  was  1.6  ±0.2  mm. 
From  our  success  with  VV  prostate  registration,  we  decided 
that  we  could  measure  SV  accuracy  by  comparing  results  to 
VV  registrations  for  those  volume  pairs  having  low  VV 
registration  error.  To  compare  SV  and  W  registration,  we 
defined  a  rectangular  volume  of  interest  (VOI)  just  covering 
the  prostate  and  calculated  voxel  displacements  between  the 
two  registrations.  To  voxels  within  the  VOI,  we  applied  the 
transformations  obtained  by  W  and  by  SV  registrations.  We 
then  calculated  the  3D  Euclidian  displacements  between  the 
transformed  voxels.  The  mean  voxel  distance  was  used  as  our 
metric  of  SV  registration  error.  For  the  evaluation  of 
algorithm  robustness,  we  defined  the  SV  registration  as  being 
successful  when  the  3D  displacement  was  less  than  2.0  mm. 

The  success  of  computer  registration  of  SPECT  and  MRI 
volumes  was  determined  by  comparing  results  to  manual 
registration.  Manual  registration  was  done  by  two  board- 
certified  nuclear  medicine  radiologists  blinded  to  the 
automatic  registration  results.  Manual  registration  was  done 
using  a  software  package  with  a  graphical  user  interface 
(GUI)  developed  in-house,  which  allows  graphical 
manipulation  of  volumes  with  six  degrees  of  freedom  in  a 
rigid  body  registration.  A  color  overlay  was  used  to  assess 
registration  quality. 

Two  radiologists  with  a  nuclear  medicine  specialty  aligned 
the  image  volumes,  and  whenever  there  was  a  discrepancy, 
they  reached  a  consensus  for  a  single  transformation.  This 
painstaking  cross-validation  was  a  time-consuming  process 
and  certainly  would  not  be  a  routine  procedure,  but  the  results 
served  as  the  gold  standard  for  the  automated  method.  We 
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defined  a  successful  automatic  registration  to  be  obtained 
when  all  displacements  were  <  2  voxels  (6  mm)  in  the  x,  y, 
and  z  directions  and  angle  differences  were  <  2  degree  for  all 
angles  about  each  of  the  three  axes. 

Although  manual  registration  is  difficult  and  somewhat 
operator  dependent,  it  is  the  only  acceptable  option  for  an 
independent  registration  on  the  patient  SPECT  and  MRI 
volumes.  Skin  fiducials  would  be  of  limited  value  in  the 
pelvis,  and  there  are  no  good  identifiable  point  anatomical 
landmarks  in  the  SPECT  images. 

We  simulated  the  iMRI-guided  procedures  using  our  image 
registration  and  fusion  software  that  are  specially  designed  for 
this  application.  Before  treatment,  we  acquired  SPECT  and 
high  resolution  MRI  volumes  from  the  same  patients.  Second, 
we  registered  the  two  images  and  transferred  the  pair  of 
aligned  data  sets  to  a  workstation  that  was  used  for  the  slice  to 
volume  registration.  Third,  we  connected  the  workstation  to 
the  iMRI  scanner  and  obtained  iMRI  image  slices  from  the 
scanner.  Fourth,  we  performed  the  slice  to  volume 
registration.  Finally,  the  software  created  fused  images  of  the 
three  modalities  as  would  be  done  for  image  guidance.  All 
registrations  and  image  fusions  are  automatic. 


IV.  Results 

A.  Registration  of  an  iMRI  Slice  to  a  High  Resolution  MRI 

Volume 

We  determined  SV  registration  results  for  slices  near  the 
prostate  in  the  three  standard  orthogonal  orientations. 
Comparing  to  VV,  mean  and  standard  deviation  registration 
errors  across  9  volume  pairs  were  0.4  mm  ±0.2  mm, 
0.5  mm  ±  0.2  mm,  and  2.6  mm  ±  1 .6  mm  for  transverse, 
coronal  and  sagittal  slices  covering  the  prostate,  respectively. 
Transverse  slices  worked  best  because  they  contain  many 
relatively  rigid  anatomical  structures. 

Simulation  experiments  showed  SV  registration  to  be  very 
insensitive  to  noise.  We  performed  over  100  registration 
experiments  with  noise  added  to  give  signal  to  noise  ratio 
(SNR)  ranging  from  20  to  5.  Using  the  slice  configurations 
recommended  above  (transverse  slices  near  the  prostate 
center),  we  obtained  100%  successful  registrations  (an  error  < 
2.0  mm)  for  SNR’s  =  10,  a  value  much  worse  than  the  clinical 
SNR  value  of  ~  25  on  our  iMRI  system. 

As  for  SV  registration  of  actual  iMRI  image  slices  with  a 
high-resolution  MR  volume,  the  contours  overlap  and  overlay 
images  show  that  the  prostate  matches  very  well  (Fig.  1). 
Other  visual  inspection  techniques  also  demonstrate  excellent 
registration.  Note  that  a  single  iMRI  image  was  used  to 
produce  this  registration  result. 

We  now  describe  some  aspects  of  the  implementation.  The 
time  for  an  SV  registration  was  typically  about  3  sec  on  a 
Pentium  IV,  2.4  GHz  CPU,  with  1  Gbytes  of  memory.  The 
algorithm  was  written  in  IDL  and  could  probably  be  made 
much  faster  in  a  lower  level  language  such  as  C. 


Fig.  1.  Prostate  MR  images.  The  top  is  the  high  resolution  MR  image.  The 
bottom  is  the  actual  iMRI  image;  the  rectangular  window  at  the  center  is  the 
transparency  display  of  both  images.  The  prostate  boundary  segmented  from 
high  resolution  MR  image,  matches  well  with  the  prostate  in  iMRI  image. 
Images  are  from  the  volunteer  S3. 

B.  Registration  of  SPECT  and  High  Resolution  MRI 

Volumes 

An  example  of  a  successful  automatic  registration  is  shown 
in  Fig.  2.  All  anatomical  features  including  the  bone  marrow 
in  the  femur  and  pubic  symphysis  are  well  aligned  in  the  color 
overlay.  This  MR-SPECT  volume  pair  and  four  others  were 
successfully  registered  according  to  the  criteria  defined 
earlier.  Standard  algorithm  parameters  (Section  II.C)  were 
used  with  the  lower-left  quadrant  of  the  joint  histogram  used 
for  calculating  MI.  Successful  image  registration  was 
obtained  with  images  from  patients  S4,  S5,  and  S8.  There 
were  four  other  MR-SPECT  volume  pairs  obtained  from 
patients  S6  and  S7  that  were  not  successfully  registered  with 
our  program.  In  all  four  cases,  the  MR  images  were  not 
acquired  using  our  final,  optimized  MR  sequence 
(Section  III.A).  When  we  used  the  optimized  sequence  with 
full  anatomical  coverage,  registration  was  always  successful. 
We  believe  that  automated  SPECT-MRI  registration  will  be 
feasible  on  many  patients’  images. 

We  now  report  the  registration  results  of  SPECT  and  high 
resolution  MRI  images  of  the  phantom.  Registrations  of  the 
phantom  images  were  carried  out  by  displacing  the  aligned 
image  pair  with  known  rotation  and  translations.  All 
orientations,  axial,  sagittal,  and  coronal,  were  successfully 
registered.  Other  experiments  showed  that  intensity  scaling 
and  multi-resolution  could  not  improve  the  registration  ability 
for  both  phantom  and  human  data. 

C.  Image  Fusion  and  Visualization 

We  created  image  registration  and  fusion  software  for  the 
potential  applications  in  iMRI-guided  procedures.  In  Fig.  3, 
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we  demonstrate  the  image  fusion  visualization  software  in  a 
simulation  of  clinical  usage.  SPECT  and  high  resolution  MR 
images  were  acquired,  transferred  to  a  workstation,  and 
registered  prior  to  the  “simulated”  procedure.  We  then 
simulate  acquiring  thick  iMRI  slices,  register  them  to  the  high 
resolution  volume,  and  prepare  the  visualization  in  Fig.  3.  In 
this  figure,  one  can  see  all.  The  registered  images  are  shown 
in  the  three  windows  at  the  top  line  (Fig.  3).  After 
registration,  the  program  creates  fused  images  as  displayed  at 
the  bottom. 


V.  Discussion  and  Conclusion 

This  preliminary  study  has  shown  promising  algorithmic 
results  for  bringing  nuclear  medicine,  functional  images  into 
the  interventional  MRI  suite.  Automatic  registration  of 
SPECT  and  MRI  volumes  was  always  successful  with  “good” 
MRI  volumes  obtained  using  the  optimized  acquisition 
sequence  and  covering  all  anatomy  of  interest.  Slice-to- 
volume  automatic  registration  was  even  more  successful  with 
highly  accurate,  robust  registration  obtained.  Putting  these 
two  steps  together,  a  patient’s  SPECT  images  can  be 
registered  to  a  high  resolution  MRI  volume  prior  to  an  iMRI 
procedure;  live-time  iMRI  slice  images  can  be  registered  to 
the  MRI  volume;  and,  finally,  one  can  then  display  the  live¬ 
time  iMRI  slice  image  with  the  appropriately  reformatted, 
fused  image  from  the  SPECT  and  high  resolution  MRI  image 
volumes. 

The  required  registration  accuracy  is  probably  less  than  one 
might  think.  The  live-time  iMRI  image  obtained  in  the  plane 
of  the  advancing  needle  will  always  be  used  for  guiding  a 
needle  for  intervention  or  biopsy.  The  corresponding  fused 
SPECT-MRI  and/or  high  resolution  MRI  images  will  be  used 
as  a  planning  guide.  With  proper  visualization  tools, 
interventional  radiologists  should  be  able  to  mentally  account 
for  any  small  registration  errors.  In  addition,  there  is  often 
image  evidence  of  cancer  in  MR  prostate  images  that  can 
perhaps  be  identified  with  the  aid  of  functional  images.  Such 
MR-visible  lesions  can  then  become  the  markers  for  tumor 
targeting.  Any  potential  gross  registration  errors  should  be 
easily  recognized  resulting  in  a  failure  to  include  the 
functional  image  data  in  the  iMRI  suite  but  not  in  a 
catastrophic  misguidance  of  the  therapy  needle. 

More  analysis  of  registration  error  is  possible.  The  overall 
registration  error  of  placing  a  SPECT  image  with  a  live-time 
iMRI  image  depends  upon  both  SPECT-MRI  and  the  slice-to- 
volume  errors.  The  slice  to  volume  error  for  voxels  near  the 
prostate  is  □  1.4  mm,  as  argued  elsewhere  [  1 1  ]  [  1 2] .  The 
SPECT-MRI  error  can  be  roughly  estimated  from  the 
requirements  for  acceptable  registration  (D6mm  and 
□  2  degrees).  Slice-to- volume  registration  has  an  error  less 
than  obtained  with  SPECT-MRI,  as  would  be  expected  from 
the  low  resolution  and  reduced  number  of  features  with 


SPECT.  The  automatic  SPECT-MRI  registration  error  is 
comparable  to  the  uncertainty  of  manual  registration.  (After 
all,  that  is  how  we  specified  the  requirements  for 
“acceptability.”)  Despite  such  uncertainty,  SPECT  images 
have  been  routinely  registered  with  CT  and  MR  images  at  our 
institution  to  use  ProstaScintD  for  diagnostic  studies  of 
prostate  cancer.  If  SPECT-MRI  can  be  used  for  diagnostic 
procedures  and  slice-to-volume  registration  has  much  less 
error,  then  we  anticipate  that  the  combined  process  will  fulfill 
needs  for  guidance  as  argued  in  the  last  paragraph. 

To  minimize  registration  error,  we  recommend  that  image 
data  are  obtained  under  comparable  conditions  by  keeping  a 
similar  posture  and  by  taking  clinical  measures  to  reduce 
rectal  and  bladder  filling.  Warping  registration  method  may 
be  useful  to  correct  significant  deformations  at  the  expense  of 
additional  complexity,  time,  and  possibly  robustness  [18][19]. 

Finally,  we  believe  that  it  is  quite  feasible  to  include 
previously  acquired  nuclear  medicine  SPECT  images  and 
high-resolution  MRI  data  into  iMRI-guided  minimally 
invasive  treatment  procedures.  We  are  beginning  to  explore 
this  application  in  animal  experiments. 


IEEE  Transaction  on  Nuclear  Science 


February  2004  (In  Press) 


Fig.  2.  Registration  results  of  patient  data.  The  top  three  images  show  corresponding  registered  SPECT,  high  resolution  MRI,  and  simulated  iMRI  images, 
respectively.  The  bottom  three  windows  show  the  fused  images  of  the  three  modalities,  from  left  to  right,  iMRI/MRI,  SPECT/MRI,  and  SPECT/iMRI, 
respectively.  Images  are  from  Patient  S4. 
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Fig.  3.  Simulation  experiments  with  phantom  using  registration  and  fusion  software.  The  top  three  windows  from  left  to  right  show  corresponding  registered 
SPECT,  high  resolution  MRI,  and  iMRI  images,  respectively.  The  bottom  three  windows  from  left  to  right  show  the  fused  images,  iMRI/MRI,  SPECT/MRI, 
and  SPECT/iMRI,  respectively.  Other  buttons  and  sliders  control  the  configuration  and  registration. 
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Abstract.  Nuclear  medicine  can  detect  and  localize  tumor  in  the  prostate  not 
reliably  seen  in  MR.  We  are  investigating  methods  to  combine  the  advantages  of 
SPECT  with  interventional  MRI  (iMRI)  guided  radioffequency  thermal  ablation 
of  the  prostate.  Our  approach  is  to  first  register  the  low-resolution  functional 
images  with  a  high  resolution  MR  volume.  Then,  by  combining  the  high- 
resolution  MR  image  with  live-time  iMRI  acquisitions,  we  can,  in  turn,  include 
the  functional  data  and  high-resolution  anatomic  information  into  the  iMRI 
system  for  improved  tumor  targeting.  In  this  study,  we  investigated  registration 
methods  for  combining  noisy,  thick  iMRI  image  slices  with  high-resolution  MR 
volumes.  We  compared  three  similarity  measures,  i.e.,  normalized  mutual 
information,  mutual  information,  and  correlation  coefficient;  and  three 
interpolation  methods,  i.e.,  re-normalized  sine,  tri-linear,  and  nearest  neighbor. 
Registration  experiments  showed  that  transverse  slice  images  covering  the 
prostate  work  best  with  a  registration  error  of  ~  0.5  mm  as  compared  to  our 
volume-to-volume  registration  that  was  previously  shown  to  be  quite  accurate  for 
these  image  pairs. 


1  Introduction 

Nuclear  medicine  can  detect  and  localize  tumor  in  the  prostate  not  reliably  seen  in 
MR.1  We  are  investigating  methods  to  combine  the  advantages  of  SPECT  with 
interventional  MRI  (iMRI)  guided  radiofrequency  (RF)  thermal  ablation  for  the 
treatment  of  the  prostate  cancer.  Our  idea  is  to  first  register  the  low-resolution 
functional  images  with  a  high  resolution  MRL2  Then  by  registering  the  high- 
resolution  MR  volume  with  live-time  iMRI  acquisitions,  we  can,  in  turn,  map  the 
functional  data  and  high-resolution  anatomic  information  to  iMRI  images  to  aid  tumor 
targeting.  We  previously  reported  a  method  for  the  registration  of  noisy,  thick  iMRI 
image  slices  with  high-resolution  MR  volumes  with  simulated  3  and  actual  iMRI 
images.4  In  this  report,  we  compared  three  interpolation  methods  and  three  similarity 
measures  for  this  application.  Hundreds  of  registration  experiments  were  performed 
with  12  pairs  of  MR  volume  images  acquired  from  four  healthy  volunteers. 
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2  Registration  Algorithms 

2.1  Three  Interpolation  Methods 

We  investigated  three  interpolation  methods,  i.e.,  re-normalized  sine  interpolation,5 
tri -linear,  and  nearest  neighbor. 

Let  the  original  data  set  be  Iorg  ,  the  re-formatting  data  set  I„ew .  The  conventional 
sine  interpolation  with  a  cosine  Hamming  window  is  described  as  below.5,6 

/„„,(*>  y,  z)  =  Z  ZZ  ^  ( *  < ' Y< Z)  • ■  H(x,  X ,  R)  ■  H  (y,Y ,  R)  ■  H  (z,  Z,  R) 

X  Y  Z 


where  H(a,  A,  R)  =  Sin^(fl  ^  •  {  1  +  cos[;r(a  -  A) /  R  + 1]} , 

2 K(a  -  A) 

and  X,  Y,  Z,  represent  the  coordinates  of  (7or<?);  x,  y,  z  the  coordinates  of  (Inew );  A  is  a 
symbol  representing  X,  Y,  or  Z,  and  a  represents  x,  y,  or  z;  and  R  is  the  kernel  size. 
The  Hamming  function  eliminates  problems  with  oscillatory  effects  at  discontinuities 
and  guarantees  that  the  convolution  coefficients  fall  of  to  zero  at  the  edge  of  the  sine 
kernel  (i.e.,  at  \a\=R+l,  where  R=5  in  this  study).5 

In  our  implementation,  we  used  the  re-normalized  sine  interpolation  method 
because  it  could  make  significant  improvement  in  performance  of  the  conventional 
sine  interpolation.5  We  replaced  H  in  the  above  equation  with 

Hn„(a,  A,  R)  =  H(a,  A,  R)jY,H{a,  A,  R)  =  H  (a,  A,  R)/\H(a)\. 


2.2  Three  Similarity  Measurements 


We  used  three  similarity  measures,  normalized  mutual  information  (NMI),  mutual 
information  (MI),  and  correlation  coefficient  (CC),  in  our  registration.  One  image  R  is 
the  reference ,  and  the  other  F  is  floating.  Their  mutual  information  MI  is  given 
below.7,8 


^  =  Z^^/)log 

rj 


Prf  (r  ’  £1 

PR(r)  '  PF(f) 


The  joint  probability  pRF(r,f)% nd  the  marginal  probabilities  /?fl(r)of  the  reference 
image  and  pF(f) of  the  floating  image,  can  be  estimated  from  the  normalized  joint 
intensity  histograms.  We  used  the  NMI  version  proposed  by  Maes.8 


NMI  = 


2  MI 

H(R)  +  H(F) 


where  H(R)  =  -JjpR{r)\ogpR(r)  and  //(F) =-£>,(/) log M/) 
r  f 


The  correlation  coefficient  CC  is  given  below.9 


Sine 


Tri-linear 


Nearest  Neighbor 


Fig.  1  Similarity  surfaces  are  plotted  as  a  function  of  translations  at  the  1/4  resolution  in  the 
multi-resolution  registration  process.  Two  high-resolution  MRI  volumes  were  registered,  and 
they  are  down  sampled  by  1/4  along  each  linear  dimension,  giving  a  distance  between  voxel 
centers  of  ~  5.5  mm.  From  the  optimal  parameters,  we  computed  the  similarity  values  of  the 
simulated  iMRI  and  MRI  images  as  a  function  of  translations  along  the  coronal  (anterior- 
posterior)  and  sagittal  (left-right)  axis.  From  top  to  bottom,  normalized  mutual  information 
(NMI),  mutual  information  (MI),  and  correlation  coefficient  (CC)  surfaces  are  plotted.  From 
left  to  right,  sine,  tri-linear,  and  nearest  neighbor  interpolations  are  used  to  obtain  the  floating 
images,  respectively.  The  noisey  NMI/MI  surfaces  show  a  false  global  maximum  and  many 
local  maxima.  CC  surfaces  are  much  smoother  indicating  its  suitability  for  low  resolution. 
Nearest  neighbor  has  a  flat  peak  with  a  width  of  one  voxel  in  similarity  surfaces.  Images  are 
from  volunteer  S2. 


2.3  Comparison  of  Similarity  Surfaces 

We  plot  the  similarity  surfaces  for  the  three  similarity  measures,  NMI,  MI,  and  CC,  at 
different  resolutions;  and  we  determine  their  suitability  for  SV  registration.  At  1/4 
resolution ,  we  resampled  images  so  as  to  give  1/4  number  of  the  voxels  along  each 
linear  dimension.  At  full  resolution ,  we  used  the  full  number  of  voxels.  We  plot  the 
similarity  measures  as  a  function  of  translations.  After  two  typical  high-resolution 
MR  volumes  were  registered,10  values  were  plotted  with  the  origin  as  the  optimal 
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transformation.  We  calculated  similarity  values  while  moving  the  simulated  iMRI 
image  relative  to  the  high-resolution  MR  image  along  coronal  (anterior-posterior)  and 
sagittal  (left-right)  axis.  When  obtaining  floating  images,  we  used  the  three  different 
interpolation  methods. 

At  1/4  resolution  (Fig.  1),  CC  surfaces  are  much  smoother  than  NMI  and  MI, 
which  are  noisy  and  contain  a  false  global  maximum  that  could  lead  to  a  false  answer 
and  many  local  maxima.11  From  these  figures,  we  infer  that  CC  is  better  at  low 
resolution.  Comparing  CC  surfaces  of  different  interpolations,  sine  and  tri-linear  have 
similar  surfaces,  and  tri-linear  is  better  than  nearest  neighbor.  For  this  application,  we 
chose  tri-linear  interpolation  instead  of  sine  because  it  is  much  faster  and  because  it 
has  comparable  performance.  Finally,  we  used  CC  and  tri-linear  at  low  resolution. 

At  full  resolution  (Fig.  2),  NMI  and  MI  surfaces  are  much  more  peaked  than  CC 
that  infers  good  optimization  accuracy,  but  once  again  there  is  high  frequency  noise  in 
the  NMI  and  MI  curves,  far  from  the  optimum,  that  gives  rise  to  local  maxima  that 
must  be  avoided.  Comparing  three  interpolation  methods,  sine  gave  the  sharpest  peak 
at  the  optimum;  nearest  neighbor  interpolation  gave  a  flat  peak  with  a  width  of  one 
voxel;  and  tri-linear  gave  a  result  between  the  other  two.  As  stated  above,  tri-linear  is 
much  faster  that  sine  with  similar  performance.  NMI  and  MI  have  no  significant 
difference  but  NMI  is  a  little  bit  robust  in  our  implementation.  We  chose  NMI  and  tri- 
linear  at  the  full  resolution. 


Sine  Tri-iinear  Nearest  Neighbor 


Fig.  2  Similarity  functions  are  plotted  as  a  function  of  translations  at  full  resolution.  Many 
details  are  given  in  the  legend  of  Fig.  1.  NMI  and  MI  surfaces  are  much  peaked  than  CC, 
especially  with  sine  and  tri-linear  interpolation.  The  voxel  is  isotropic  with  1.4  mm  on  a  side. 
Image  data  are  the  same  used  in  Fig.  1. 
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2.4  Combination  of  Normalized  Mutual  Information  and  Correlation  Coefficient 

As  a  result  of  the  above  analyses,  we  created  a  registration  algorithm  for  prostate  MR 
images.  We  define  the  iMRI  image  slice  to  be  the  reference  image;  the  matching  slice 
from  the  high-resolution  MRI  volume  is  the  floating  image.  We  use  a  multi-resolution 
approach  and  perform  registration  from  low  to  high  resolution.  We  use  CC  at  the  two 
lower  resolutions  because  it  gives  fewer  local  maxima  and  because  it  can  be 
calculated  faster  than  NMI.  We  use  NMI  at  full  resolution  because  of  its  peaked 
surface.  To  avoid  local  maxima,  we  include  a  restarting  feature  where  registration  is 
restarted  with  randomly  perturbed  parameters  obtained  from  a  uniform  distribution 
about  the  initial  transformation  values  at  the  current  resolution  being  used.  The 
algorithm  restarts  until  the  absolute  CC  is  above  a  threshold  of  0.5  as  experimentally 
determined  or  the  maximum  number  of  restarts  is  reached. 

For  registration,  we  use  a  rigid  body  transformation  (three  translations  and  three 
rotations).  For  optimization,  we  use  the  downhill  simplex  method  of  Nelder  and 
Mead. 12  Optimization  of  similarity  ends  either  when  the  maximum  number  of 
calculations  is  reached  (typically  500)  or  when  the  fractional  change  in  the  similarity 
function  is  smaller  than  a  tolerance  (typically  0.001).  We  use  IDL  (Interactive  Data 
Language,  Research  System  Inc.,  Boulder,  CO.)  as  the  programming  language.  We 
use  an  initial  guess  assuming  an  identity  transformation,  i.e.,  all  initial  translation  and 
rotation  parameters  are  zero,  because  the  patient  is  normally  oriented  approximately 
the  same  way  from  one  scan  to  the  next.  We  set  the  maximum  numbers  of  restarts  at 
10,  5,  and  3,  from  low  to  high  resolution,  respectively. 


2.5  Registration  Evaluation 

We  used  a  variety  of  evaluation  methods.  We  used  RegViz,  a  program  created  in  IDL 
in  our  laboratory  with  multiple  visualization  and  analysis  methods.  First,  we  manually 
segmented  prostate  boundaries  in  image  slices  and  copied  them  to  corresponding 
slices.  This  enabled  visual  determination  of  the  overlap  of  prostate  boundaries  over 
the  entire  volume.  Second,  color  overlay  displays  were  used  to  evaluate  overlap  of 
structures.  To  visualize  potential  differences,  it  was  quite  useful  to  interactively 
change  the  contribution  of  each  image  using  the  transparency  scale.  Third,  we  used  a 
sector  display,  which  divided  the  reference  and  registered  images  into  rectangular 
sectors  and  created  an  output  image  by  alternating  sectors  from  the  two  input  images. 
Even  subtle  shifts  of  edges  would  be  clearly  seen. 

Our  quantitative  evaluation  method  for  slice  to  volume  registration  was  to  compare 
SV  and  VV  registration.10  For  volume  pairs  acquired  over  a  short  time  span  from  a 
supine  subject  with  legs  flat  on  the  table,  prostates  were  well  aligned  and  prostate 
centroid  displacements  were  typically  <  1  mm.  The  registration  accuracy  as 
determined  from  displacements  of  pelvic  bony  landmarks  was  1.6  ±  0.2  mm,  a  value 
comparable  to  error  associated  with  locating  the  landmarks.10  To  compare  SV  and  W 
registration,  we  defined  a  rectangular  volume  of  interest  (VOI)  just  covering  the 
prostate  and  calculated  voxel  displacements  between  the  two  registrations.  We 
defined  the  SV  registration  as  being  successful  when  the  3D  displacement  was  less 
than  2.0  mm. 
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3  Experimental  Methods 

3.1  Imaging  Experiments 

We  acquired  high  resolution  MRI  volumes  from  a  1.5  T  Siemens  MRI  system 
(Magnetom  Symphony,  Siemens  Medical  Systems,  Erlangen,  Germany).  An  8- 
element  phased  array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a 
uniform  sensitivity.  We  used  a  3D  rapid  gradient  echo  sequence  (PSIF)  designed  to 
acquire  the  spin-echo  component  of  the  steady  state  response,  rather  then  the  free 
induction  decay.  The  sequence  with  9.4/5.0/60  (TR/TE/flip)  yielded  160  x  256  x  128 
voxels  over  a  219  x  350  x  192-mm  rectangular  FOV  and  1.4  x  1.4  x  1.5-mm  voxels 
oriented  to  give  the  highest  resolution  for  transverse  slices.  There  was  over  sampling 
at  31%  in  the  slice  direction  to  reduce  aliasing  artifacts.  The  acquisition  time  was  4.3 
min.  The  sequence  gave  excellent  image  contrast  for  the  prostate  and  its  surroundings. 

We  acquired  high  resolution  MRI  volumes  from  four  volunteers  S1-S4.  For  each 
volunteer,  three  image  volumes  were  obtained  with  an  imaging  session.  Each  volume 
was  acquired  with  compatible  conditions.  Volunteers  laid  supine  with  legs  flat  similar 
to  the  position  in  routine  MR  scanning.  Between  volume  acquisitions,  volunteers  got 
off  the  MRI  table,  stretched,  and  walked  around  to  ensure  that  they  would  assume  a 
different  position  when  they  laid  back  on  the  table.  The  coil  array  was  centered  on  the 
prostate.  All  images  of  a  volunteer  were  acquired  with  the  same  MRI  acquisition 
parameters.  In  total,  there  are  12  pairs  of  high-resolution  MRI  volumes  for 
registration. 

We  used  the  high-resolution  MRI  volumes  to  simulate  iMRI  images  by  creating 
thick  slices  and  adding  noise.  MR  noise  is  described  by  the  Rician  distribution,13  but 
at  reasonably  high  signal  values,  the  noise  is  accurately  approximated  with  Gaussian 
white  noise.14  We  added  Gaussian  noise  to  the  simulated  iMRI  slice  images. 
Clinically,  we  typically  use  an  iMRI  slice  thickness  of  4.0  -  6.0  mm.  We  averaged  3 
1.4  mm  thick  slices  to  create  a  4.2  mm  thick  slice. 

Additionally,  we  acquired  real  iMRI  images  from  volunteers  SI -S3  using  a  clinical 
0.2  T  C-arm  open  MR  scanner  (Siemens  Open  Symphony,  Erlangen,  Germany).  We 
used  a  two-dimensional  (2D)  PSIF  sequence  with  15.2/7.4/45  (TR/TE/FA)  for  image 
slice  acquisitions.  The  iMRI  slices  were  128x128  with  in-plane  pixel  size  of 
2.8x2. 8  mm  and  with  effective  slice  thickness  of  5  mm. 


3.2  Registration  Experiments 

We  used  12  pairs  of  high-resolution  MR  volumes  to  perform  registration  experiments. 
For  each  volume  pair,  we  extracted  data  from  one  volume  to  simulate  thick  iMRI 
image  slices;  and  then  we  registered  the  simulated  image  slices  to  the  other  volume. 
We  desire  an  iMRI  slice  image  acquisition  method  that  gives  robust,  accurate 
registrations  and  is  relatively  insensitive  to  acquisition  parameters.  Hence,  we 
performed  experiments  to  determine  the  dependence  on  slice  orientation  (transverse, 
sagittal  and  coronal),  on  slice  position  relative  to  the  prostate  (above,  centered,  and 
below)  and  on  image  noise  from  fast  imaging  techniques. 

We  also  performed  SV  registration  experiments  using  actual  iMRI  images.  We 
registered  actual  iMRI  image  slices  with  high-resolution  (1.5  T  system)  MR  volumes 
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and  visually  evaluated  results.  For  each  volunteer  SI -S3,  there  were  three  high- 
resolution  MR  volumes  and  30  iMRI  image  slices  giving  90  SV  registration 
experiments,  and  a  total  of  270  experiments. 


Fig.  3  Prostate  images  of  high  resolution  MRI  (a)  and  interventional  MRI  (b).  The  rectangular 
region  at  the  center  of  image  (c)  is  the  overlay  display  of  both  images.  The  prostate  matches 
well.  Images  are  from  S3. 


4  Results 

4.1  Simulated  Images 

Using  simulated  iMRI  images,  we  determined  SV  registration  results  for  slices  near 
the  prostate  in  the  three  standard  orthogonal  orientations.  Comparing  to  VV,  average 
registration  errors  were  0.4  mm,  0.5  mm,  and  2.6  mm  for  transverse,  coronal  and 
sagittal  slices  covering  the  prostate,  respectively.  Transverse  slices  worked  best 
because  they  contained  many  relatively  rigid  anatomical  structures.  Coronal  slices 
worked  next  best.  Sagittal  slices  gave  the  largest  error  because  they  contained  a  large 
portion  of  the  deformable  bladder  and  rectum. 
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The  registration  is  insensitive  to  noise.  Typical  iMRI  SNR  under  clinical  conditions 
is  about  25.  Even  when  noise  much  exceeded  this  normal  situation,  registration  results 
were  quite  good.  A  100%  success  rate  was  achieved  with  an  acceptance  criterion  of 
<  2.0  mm  even  when  SNR  was  as  bad  as  10. 


4.2  Actual  iMRI  Images 

Registration  of  actual  iMRI  image  slices  with  a  high-resolution  MR  volume  was 
successful.  The  contours  overlap  and  overlay  images  show  that  the  prostate  matches 
very  well.  Other  visual  inspections  also  demonstrate  excellent  registration.  Note  that  a 
single  iMRI  image  was  used  to  produce  this  registration  result. 


4.3  Algorithm  Implementation 

Computation  time  and  registration  accuracy  are  two  main  factors  to  consider  when 
choosing  interpolation  methods.  Using  tri-linear  interpolation,  the  time  for  an  SV 
registration  was  typically  about  5  sec  on  a  Pentium  IV,  1.8  GHz  CPU,  with  1Gbyte  of 
memory.  When  the  re-normalized  sine  interpolation  method  was  used,  the  time  was  ~ 
10  min,  a  duration  not  acceptable  for  our  application.  The  algorithm  was  written  in 
IDL  and  could  probably  be  made  faster  in  a  lower  level  language  such  as  C.  We  did 
not  use  nearest  neighbor  because  of  insufficient  accuracy  as  deduced  from  its  flat 
peak  of  the  similarity  surfaces  in  Figure  2.  A  call  to  the  Simplex  optimization 
typically  resulted  in  50  to  150  similarity  evaluations  before  the  tolerance  value 
(0.001)  was  reached. 


4  Discussion  and  Conclusion 

The  comparison  of  similarity  surfaces  enabled  us  to  design  a  robust,  fast,  and  accurate 
registration  algorithm  for  the  potential  applications  of  iMRI-guided  thermal  ablation 
of  the  prostate  cancer.  A  single  iMRI  image  slice  achieved  nearly  the  same  accuracy 
as  obtained  from  volume-to- volume  registration.  Since  live-time  iMRI  images  are 
used  for  guidance  and  registered  images  are  used  for  adjunctive  information,  the 
registration  accuracy  is  very  probably  adequate.  As  compared  to  a  typical  SPECT 
and/or  iMRI  slice  thickness  of  >  3.0  mm,  SV  registration  is  quite  accurate. 

If  one  were  to  use  functional  or  high-resolution  MR  images  directly  for  tumor 
targeting  within  the  relatively  small  prostate,  the  requirements  for  registration 
accuracy  would  be  great.  However,  fused  image  data  will  not  be  used  blindly.  Rather, 
these  visualizations  will  be  used  as  a  guide.  Physicians  will  always  use  the  live-time 
iMRI  images  for  needle  guidance.  With  proper  visualization  tools,  physicans  should 
be  able  to  mentally  account  for  any  small  registration  errors.  Moreover,  the  functional 
images  might  enable  one  to  find  cancer  features  in  the  iMRI  images. 

Finally,  we  conclude  that  it  is  quite  feasible  to  include  previously  acquired  high- 
resolution  MRI  and  nuclear  images  into  iMRI-guided  treatment  procedures.  We  are 
beginning  to  explore  this  application  in  animal  experiments. 
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Abstract.  We  are  investigating  interventional  MRI  (iMRI)  guided  thermal 
ablation  treatment  of  the  prostate  cancer.  Functional  images  such  as  SPECT  can 
detect  and  localize  tumor  in  the  prostate  not  reliably  seen  in  MRI.  We  intend  to 
combine  the  advantages  of  SPECT  with  iMRI-guided  treatments.  Our  concept  is 
to  first  register  the  low-resolution  SPECT  with  a  high  resolution  MRI  volume. 
Then  by  registering  the  high-resolution  MR  image  with  iMRI  acquisitions,  we 
can,  in  turn,  map  the  functional  data  and  high-resolution  anatomic  information  to 
iMRI  images  for  improved  tumor  targeting.  For  the  first  step,  we  used  a  mutual 
information  registration  method.  For  the  latter,  we  developed  a  robust  slice  to 
volume  (SV)  registration  algorithm.  Image  data  were  acquired  from  patients  and 
volunteers.  Compared  to  our  volume-to-volume  registration  that  was  previously 
evaluated  to  be  quite  accurate,  the  SV  registration  accuracy  is  about  0.5  mm  for 
transverse  images  covering  the  prostate.  With  our  image  registration  and  fusion 
software,  simulation  experiments  show  that  it  is  feasible  to  incorporate  SPECT 
and  high  resolution  MRI  into  the  iMRI-guided  treatment. 


1  Introduction 

We  use  an  interventional  magnetic  resonance  imaging  (iMRI)  system  to  guide 
minimally  invasive  treatments,  including  the  radiofrequency  (RF)  thermal  ablation  of 
abdominal  cancers.  13  The  iMRI  system  consists  of  a  0.2  T,  clinical  C-arm  open  MRI 
scanner,  an  in-room  RF-shielded  liquid  crystal  monitor,  an  MR  compatible  mouse,  a 
foot  pedal,  and  a  RF  device.  We  are  currently  investigating  the  extension  of  these 
techniques  to  the  treatment  of  prostate  cancer.  Since  MRI  does  not  reliably  show 
prostate  tumors,  we  intend  to  incorporate  nuclear  medicine  images  with  higher 
sensitivity  for  detecting  and  localizing  prostate  tumors  4,5  We  will  first  register  the 
low-resolution  functional  SPECT  images  with  a  high  resolution  MRI  volume.  Then 
by  registering  the  high-resolution  MR  volume  with  iMRI  acquisitions,  we  can,  in  turn, 
map  the  functional  data  and  high-resolution  anatomic  information  to  iMRI  images  for 
improved  tumor  targeting.  If  this  procedure  is  successful,  then  a  variety  of  potential 
visualization  tools  can  help  the  physician  appropriately  localize  and  apply  treatments. 
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In  next  sections,  we  will  report  a  three  dimensional  registration  method  for  SPECT 
and  high  resolution  MRI  volumes,  a  slice  to  volume  registration  algorithm  for  iMRI 
thick  slices  and  high  resolution  MRI  volume,  and  image  registration  and  fusion 
software  for  potential  applications  in  iMRI-guided  thermal  ablation  of  the  prostate. 


2  Registration  Algorithms 

2.1  Registration  of  SPECT  and  High  Resolution  MRI  Volumes 

A  mutual  information  algorithm  was  applied  in  this  study  to  the  registration  of  pelvic 
image  volumes  from  MRI  and  SPECT  for  potential  use  in  prostate  cancer  diagnosis, 
staging,  and  treatment  planning.  Mutual  information  (MI)  was  chosen  because  of  its 
potential  to  align  multi-modality  images6,7.  Registration  of  SPECT  and  MR  images  is 
challenging  because  the  two  image  types  have  very  different  spatial  resolutions  and 
image  features.  The  pelvic  region  is  difficult  for  alignment  between  images  from  the 
two  scans.  Before  registration,  both  SPECT  and  MRI  volumes  were  resized  using  tri- 
linear  interpolation  to  create  volumes  matrix  of  128x128x128  with  3  mm  isotropic 
voxels,  a  voxel  size  between  that  of  the  two  scans.  The  standard  parameter  set  for 
automatic  registration  include:  256  intensity  levels  for  each  volume,  the  entire  2D 
joint  histogram,  the  full  field  of  view  of  128x128x128  voxels  for  both  volumes,  and 
no  masking  or  cropping  of  either  volume. 

2.2  Registration  of  iMRI  Slice  and  High  Resolution  MRI  Volume 

We  used  two  similarity  measures,  mutual  information8  and  correlation  coefficient 
(CC),9  in  our  registration.  We  used  a  similar  algorithm  as  previously  reported  by  us.10 
We  use  a  multi-resolution  approach  and  perform  registration  from  low  to  high 
resolution.  We  use  correlation  coefficient  at  the  two  lower  resolutions  because  it  gives 
fewer  local  maximums  and  because  it  can  be  calculated  faster  than  MI.11  We  use  MI 
at  full  resolution  because  the  peaked  similarity  function  gives  a  more  precise  solution 
than  CC.12  To  avoid  local  maximums,  we  include  a  restarting  feature  where 
registration  is  restarted  with  randomly  perturbed  parameters  obtained  from  a  uniform 
distribution  about  the  initial  transformation  values  at  the  current  resolution  being 
used.13  The  algorithm  restarts  until  the  absolute  CC  is  above  an  experimentally 
determined  threshold  or  the  maximum  number  of  restarts  is  reached. 

3  Experimental  Methods 

3.1  SPECT  Image  Acquisition 

We  acquired  SPECT  images  from  three  patients  SI -S3  with  either  high  Gleason 
scores  (>5)  from  biopsy  or  rising  PSA  level  (>10  mcg/L,  prostate  specific  antigen)  or 
palpatation  staging  beyond  stage  Tl.  After  patient  eligibility  was  established,  patients 
gave  informed  consent.  The  Institutional  Review  Board  of  the  University  Hospitals  of 
Cleveland  approved  the  imaging  protocol.  The  radiotracer  used  for  SPECT  imaging  is 
ProstaScint®  (Cytogen  Corporation,  Princeton,  NJ),  a  monoclonal  antibody  that  binds 
to  prostate-specific  membrane  antigen  (PSMA). 


Approximately  four  days  after  injecting  5  mCi  ProstaScint®,  the  abdominal  region 
and  pelvic  region  were  scanned  using  a  two-head  Siemens  E.CAM*"  camera  (Siemens 
Medical  System,  Inc.,  Hoffman  Estates,  IL).  The  evening  before  the  scanning, 
patients  were  asked  to  perform  a  bowel  prep  with  Fleet  R  Prep  Kit  #3  (Fleet 
Pharmaceuticals,  Lynchburg,  VA).  Images  were  acquired  with  a  medium  energy 
collimator  and  1 5%  energy  window.  The  acquisition  parameters  included  a  step-and- 
shoot  motion,  a  128  x  128  pixel  matrix  for  each  projection,  an  imaging  time  of  25  sec 
per  stop,  and  a  total  of  120  stops  over  a  full  360°  rotation.  The  field  of  view  of  was 
53.3  x  38.7-cm.  The  iterative  image  reconstruction  algorithm  OSEM  (ordered  subsets 
expectation  maximization) 14  was  used  to  reconstruct  the  SPECT  images  with  8 
subsets  and  10  iterations.  The  SPECT  images  were  comprised  of  isotropic  voxels  with 
size  of  4.8x4.8x4.8-mm.  Each  patient  had  one  SPECT  scan  of  the  pelvis. 

3.2  High  Resolution  MR  Image  Acquisitions 

High-resolution  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system 
(Magnetom  Symphony,  Siemens  Medical  Systems,  Erlangen,  Germany).  An  8- 
element  phased  array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a 
uniform  sensitivity.  Typically  two  anterior  and  two  posterior  elements  were  enabled 
for  signal  acquisition.  We  used  two  different  MR  sequences. 

First,  we  used  a  3D  rapid  gradient  echo  sequence  (PSIF)  designed  to  acquire  the 
spin-echo  component  of  the  steady  state  response,  rather  than  the  free  induction 
decay.  The  spin  echo  component  forms  immediately  prior  to  the  RF  pulse;  it  is  shifted 
toward  the  prior  RF  pulse  through  appropriate  gradient  waveform  design.  The 
sequence  with  9.4/5.0/60  (TR/TE/flip)  yields  160x256x128  voxels  over  a 
219x350xl92-mm  rectangular  FOV.  There  is  over  sampling  at  31%  in  the  slice 
direction  to  reduce  aliasing  artifacts.  The  acquisition  time  is  4  min  and  15  sec.  This 
sequence  gave  excellent  image  contrast  for  the  prostate  and  its  surroundings.  It  was 
used  to  acquire  volumes  for  volunteers  S4-S6.  Second,  we  used  a  3D  RF  spoiled 
gradient  echo  steady  state  pulse  sequence  (FLASH)  with  TR/TE/flip  parameters  of 
12/5.0/60  which  give  256  x  256  x  128  voxels  over  a  330x330x256-mm  field  of  view 
(FOV)  to  yield  1.3xl.3x2.0-mm  voxels  oriented  to  give  the  highest  resolution  for 
transverse  slices.  The  acquisition  time  is  5  min  and  38  sec.  This  sequence  is  good  for 
pelvic  imaging  but  is  not  ideal  for  the  prostate.  It  was  used  to  acquire  volumes  for 
patients  SI -S3. 

When  acquiring  MR  volumes,  volunteers  laid  supine  in  a  manner  similar  to  the 
diagnostic  position  in  routine  MR  scanning.  Between  volume  acquisitions,  volunteers 
got  up  from  the  MR  table,  stretched,  and  walked  around  to  ensure  that  they  would 
assume  a  different  position  when  they  laid  back  on  the  table.  The  coil  array  was 
centered  on  the  prostate.  We  acquire  one  volume  from  each  of  patients  SI -S3  and 
three  volumes  from  each  of  volunteers  S4-S6. 

3.3  Interventional  MRI  Image  Acquisitions  and  Simulation 

We  acquired  iMRI  images  using  a  clinical  0.2  T  C-arm  open  MR  scanner  (Siemens 
Open  Symphony,  Erlangen,  Germany)  modified  for  interventional  MRI  procedures 
and  in  this  paper  referred  to  as  the  iMRI  system.  We  used  a  3D  PSIF  with  25/13/60 
(TR/TE/FA)  for  image  volume  acquisitions  and  two-dimensional  (2D)  PSIF  with 


15.2/7.4/45  (TR/TE/FA)  for  image  slice  acquisitions.  The  iMRI  volumes  were 
256x256x100  with  voxel  size  of  1.3x1. 3x1. 4  mm.  The  iMRI  slices  were  128x128 
with  in-plane  pixel  size  of  2.8x2.8  mm  and  with  effective  slice  thickness  of  5  mm. 

We  acquired  iMRI  images  under  the  conditions  simulating  the  treatment 
application.  The  volunteer  was  supine,  and  his  legs  were  supported  at  30°-60°  relative 
to  the  horizon  and  separated  in  a  “V”  with  an  angle  of  60°-90°  between  two  legs.  This 
is  similar  to  the  lithotomy  position  used  in  prostate  therapies,  and  it  should  provide 
access  for  needle  insertion  in  brachytherapy  or  RF  thermal  ablation.  We  call  this  the 
treatment  position.  For  each  of  the  volunteers  S4-S6,  we  acquired  two  volumes  and  30 
iMRI  image  slices  covering  the  prostate.  They  included  10  transverse,  10  coronal,  and 
10  sagittal  image  slices.  We  call  these  images  “actual”  iMRI  images  to  differentiate 
them  from  later  experiments  using  “simulated”  iMRI  slices. 

To  test  a  variety  of  clinical  conditions,  we  used  high-resolution  MRI  volumes  to 
simulate  iMRI  images  by  creating  thick  slices  and  adding  noise  and  receive  coil 
inhomogeneity.  Clinically,  we  typically  use  an  iMRI  slice  thickness  of  4.0  -  6.0  mm. 
We  averaged  3  slices  1.4  mm  thick  to  create  a  4.2  mm  thick  slice.  We  added  noise  to 
the  simulated  iMRI  image.15,16 

3.4  Registration  and  Fusion  Experiments 

We  performed  three  dimensional  MI  registration  of  MRI  and  SPECT  images  using 
three  pairs  of  data  sets  from  patients  S1-S3,  Before  registration,  we  preprocessed  both 
MR  and  SPECT  images  such  as  intensity  scaling  and  isotropic  processing. 

We  used  nine  pairs  of  high-resolution  MR  volumes  to  perform  simulated  slice  to 
volume  registration  experiments.  For  each  volume  pair,  we  extracted  data  from  one 
volume  to  simulate  thick  iMRI  image  slices;  and  then  we  registered  the  simulated 
image  slices  to  the  other  volume.  We  desire  an  iMRI  slice  image  acquisition  method 
that  gives  robust,  accurate  registrations  and  is  relatively  insensitive  to  acquisition 
parameters.  Hence,  we  performed  experiments  to  determine  the  dependence  on  slice 
orientation  (transverse,  sagittal  and  coronal),  on  slice  position  relative  to  the  prostate 
(above,  centered,  and  below)  and  on  image  noise  from  fast  imaging  techniques. 

We  performed  SV  registration  experiments  using  actual  iMRI  images.  We  visually 
evaluated  results.  For  each  volunteer  S4-S6,  there  were  three  high-resolution  MR 
volumes  and  30  iMRI  image  slices  giving  90  SV  registration  experiments,  and  a  total 
of  270  experiments. 

We  simulated  the  iMRI-guided  procedures  using  our  image  registration  and  fusion 
software  that  are  specially  designed  for  this  application.  Before  treatment,  we 
acquired  SPECT  and  high  resolution  MRI  volumes  from  the  same  patient.  Second,  we 
registered  the  two  images  and  transferred  the  pair  of  aligned  data  sets  to  a  workstation 
that  was  used  for  slice  to  volume  registration.  Third,  we  connected  the  workstation  to 
the  iMRI  scanner  and  obtained  iMRI  image  slices  from  the  scanner.  Fourth,  we 
performed  slice  to  volume  registration.  Finally,  the  software  created  fused  images  of 
the  three  modalities  for  image  guidance.  All  registration  and  fusion  are  automatic. 

3.5  Registration  Evaluation 

Although  we  evaluated  registration  using  a  variety  of  quanitative  measures,  visual 
inspection  was  also  employed.  We  used  RegViz ,  a  program  created  in  IDL  in  our 


laboratory  with  multiple  visualization  and  analysis  methods.  First,  we  manually 
segmented  prostate  boundaries  in  image  slices  and  copied  them  to  corresponding 
slices.  This  enabled  visual  determination  of  the  overlap  of  prostate  boundaries  over 
the  entire  volume.  Second,  color  overlay  displays  were  used  to  evaluate  overlap  of 
structures.  To  visualize  potential  differences,  it  was  quite  useful  to  interactively 
change  the  contribution  of  each  image  using  the  transparency  scale.  Third,  we  used  a 
sector  display,  which  divided  the  reference  and  registered  images  into  rectangular 
sectors  and  created  an  output  image  by  alternating  sectors  from  the  two  input  images. 
Even  subtle  shifts  of  edges  would  be  clearly  seen. 

Registration  between  SPECT  and  MRI  was  devaluated  by  comparing  automatic 
registration  results  to  manual  registration,  our  gold  standard.  Manual  registration  was 
done  by  two  board-certified  nuclear  medicine  radiologists  who  were  blinded  to  the 
automatic  registration  results  prior  to  performing  registration.  Manual  registration  was 
done  using  a  software  package  with  a  graphical  user  interface  (GUI)  developed  in- 
house,  which  allows  graphical  manipulation  of  volumes  with  six  degrees  of  freedom 
in  a  rigid  body  registration.  A  successful  automatic  registration  is  defined  as  when  the 
transformation  parameters  are  almost  equal  to  that  of  the  manual  registration. 
Displacements  must  be  <  2  voxels  (6  mm)  in  the  x,  y,  or  z  directions  and  <  2  degree 
in  rotation  for  the  three  angles  measured  about  each  of  the  3  axes. 

The  standard  evaluation  method  for  the  slice  to  volume  registration  was  to  compare 
SV  and  volume  to  volume  (W)  registration.17  It  is  quite  reasonable  to  use  VV 
registration  as  our  gold  standard  because  it  was  previously  identified  to  be  quite 
accurate.  For  volume  pairs  acquired  over  a  short  time  span  from  a  supine  subject  with 
legs  flat  on  the  table,  following  VV  registration,  prostates  were  well  aligned  and 
prostate  centroid  displacements  typically  <  1  mm.  The  VV  registration  accuracy  as 
determined  from  displacements  of  pelvic  bony  landmarks  was  1.6  ±0.2  mm.  To 
compare  SV  and  VV  registration,  we  defined  a  rectangular  volume  of  interest  (VOI) 
just  covering  the  prostate  and  calculated  voxel  displacements  between  the  two 
registrations.  We  report  the  average  voxel  displacement  as  an  error  measure.  In 
addition,  we  defined  the  SV  registration  as  being  successful  when  the  3D 
displacement  was  less  than  2.0  mm. 

4  Results 

4.1  Registration  of  iMRI  Slice  to  High  Resolution  MRI  Volume 

We  reported  SV  registration  results  for  slices  near  the  prostate  in  the  three  standard 
orthogonal  orientations.  Comparing  to  VV,  mean  and  standard  deviation  registration 
errors  across  9  volume  pairs  and  60  SV  registration  experiments  were  0.4  mm  ±  0.2 
mm,  0.5  mm  ±0.2  mm,  and  2.6  mm  ±  1.6  mm  for  transverse,  coronal  and  sagittal 
slices  covering  the  prostate,  respectively.  Transverse  slices  worked  best  because  they 
contain  many  relatively  rigid  anatomical  structures. 

Registration  experiments  with  simulated  iMRI  images  showed  the  SV  registration 
was  very  insensitive  to  noise.  We  performed  over  100  registration  experiments  with 
noise  added  to  give  signal  to  noise  ratio  (SNR)  ranging  from  20  to  5.  Using  the  slice 
configurations  recommended  above  (transverse  slices  near  the  prostate  center),  we 


obtained  100%  successful  registrations  (an  error  <  2.0  mm)  for  SNR’s  =  10,  a  value 
much  worse  than  the  clinical  SNR  value  of  ~  25  on  our  iMRI  system. 

For  registration  of  actual  iMRI  image  slices  and  a  high-resolution  MR  volume,  the 
contours  overlap  and  overlay  images  show  that  the  prostate  matches  very  well.  Other 
visual  inspection  techniques  also  demonstrate  excellent  registration.  Note  that  a  single 
iMRI  image  was  used  to  produce  this  registration  result. 

The  time  for  an  SV  registration  was  typically  about  5  sec  on  a  Pentium  IV, 
1.8  GHz  CPU,  with  1  Gbytes  of  memory.  The  algorithm  was  written  in  IDL 
(Interactive  Data  Language,  Research  System  Inc.,  CO)  and  could  probably  be  made 
much  faster  in  a  lower  level  language  such  as  C. 
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Fig.  1.  User  interface  of  the  image  registration  and  fusion  software.  The  top  three  windows 
from  left  to  right  show  corresponding  registered  SPECT,  high  resolution  MRI,  and  iMRI 
images,  respectively.  The  bottom  three  windows  from  left  to  right  show  the  fused  images, 
iMRI/MRI,  SPECT/MRI,  and  SPECT/iMRI,  respectively.  Other  buttons  and  sliders  control 
the  configuration  and  registration.  High  resolution  MRI  and  SPECT  images  are  from  the 
patient  SI  and  the  iMRI  image  was  simulated  from  the  high  resolution  MRI  image. 


4.2  Registration  Results  of  SPECT  and  High  Resolution  MRI  Volumes 

In  Figure  1 ,  we  show  an  example  of  a  successful  registration  of  SPECT  and  MRI.  All 
anatomical  features  including  the  bone  marrow  in  the  femur  and  pubic  symphysis  are 
well  aligned  in  the  color  overlay.  Using  the  standard  algorithm  in  Section  2.1,  the  MI 
program  successfully  registered  the  three  volume  pairs.  The  standard  parameter  set 
was  defined  above  with  the  only  exception  that  the  lower-left  quadrant  of  the  joint 


histogram  was  used  for  calculating  MI.  A  successful  registration  was  judged  by  the 
criteria  defined  previously. 

4.3  Image  Fusion  and  Visualization 

In  Fig.  1,  we  demonstrate  the  image  fusion  visualization  software  in  a  simulation  of 
clinical  usage.  SPECT  and  high  resolution  MR  images  were  acquired,  transferred  to  a 
workstation,  and  registered  prior  to  the  “simulated”  procedure.  We  then  simulate 
acquiring  thick  iMRI  slices,  register  them  to  the  high  resolution  volume,  and  prepare 
the  visualization.  In  this  figure,  one  can  see  all.  The  registered  images  are  shown  in 
the  three  windows  at  the  top  line.  After  registration,  the  program  creates  fused  images 
as  displayed  in  the  bottom  windows. 

5  Discussion  and  Conclusion 

The  automatic  slice  to  volume  registration  algorithm  is  quite  robust  for  transverse 
slice  images  covering  the  prostate  and  the  registration  accuracy  is  probably 
sufficiently  accurate  to  aid  iMRI-guided  thermal  ablation  treatment.  In  clinical 
applications,  physicians  will  always  use  the  live-time  iMRI  images  for  needle 
guidance.  With  fused  image  data  and  visualization  tools,  they  should  be  able  to 
mentally  account  for  any  small  registration  errors.  In  addition,  very  often  there  is 
image  evidence  of  cancer  in  MR  prostate  images  that  can  perhaps  be  identified  with 
the  aid  of  functional  images. 

We  recommend  that  images  are  obtained  under  comparable  conditions  by  keeping 
a  similar  posture  and  by  taking  clinical  measures  to  reduce  rectal  and  bladder  filling. 
For  images  acquired  in  quite  different  positions  with  significant  deformation,  we 
previously  reported  a  warping  registration  method18  that  can  correct  the  deformations 
at  the  expense  of  additional  complexity,  time,  and  possibly  robustness. 

We  think  that  it  is  feasible  to  include  previously  acquired  high-resolution  MRI  and 
nuclear  images  into  iMRI-guided  treatment  procedures.  Image  registration  and  fusion 
would  provide  a  useful  tool  for  the  image-guided  application. 
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Abstract 

A  three-dimensional  warping  registration  algorithm  was  created  and  compared  to  rigid  body  registration  of  magnetic  resonance  (MR) 
pelvic  volumes  including  the  prostate.  The  rigid  body  registration  method  combines  the  advantages  of  mutual  information  (MI)  and 
correlation  coefficient  at  different  resolutions.  Warping  registration  is  based  upon  independent  optimization  of  many  interactively  placed 
control  points  (CP’s)  using  MI  and  a  thin  plate  spline  transformation.  More  than  100  registration  experiments  with  17  MR  volume  pairs 
determined  the  quality  of  registration  under  conditions  simulating  potential  interventional  MRI-guided  treatments  of  prostate  cancer.  For 
image  pairs  that  stress  rigid  body  registration  (e.g.  supine,  the  diagnostic  position,  and  legs  raised,  the  treatment  position),  both  visual  and 
numerical  evaluation  methods  showed  that  warping  consistently  worked  better  than  rigid  body.  Experiments  showed  that  —  1 80  strategically 
placed  CP’s  were  sufficiently  expressive  to  capture  important  features  of  the  deformation. 

©  2002  Elsevier  Science  Ltd.  All  rights  reserved. 
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1.  Introduction 

We  are  investigating  three-dimensional  (3D)  image 
registration,  particularly  with  regard  to  minimally  invasive, 
interventional  magnetic  resonance  imaging  (iMRI)  guided 
treatment  of  prostate  cancer.  At  our  institution,  we  currently 
use  iMRI  on  a  low-field  open  magnet  system  to  guide 
radiofrequency  (RF)  thermal  ablation  of  abdominal  cancer 
[1,2],  and  we  are  investigating  this  method  for  prostate 
cancer  treatment.  A  unique  feature  of  iMRI-guided  thermal 
ablation  is  that  therapy  can  be  monitored  with  MR  either  by 
acquiring  images  of  the  thermally  induced  lesion  or  by 
measuring  temperature.  In  addition,  MR  imaging  of  the 
prostate  is  desirable  because  it  more  accurately  delineates 
the  prostate  than  does  CT  [3],  which  can  overestimate  the 
prostate  volume  [4],  and  ultrasound,  which  has  a  tendency 
to  underestimate  the  extent  of  lesions  [5]. 

Several  applications  in  prostate  cancer  diagnosis, 
staging,  and  therapy  require  registration  of  MR  volumes 
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and/or  volumes  from  other  imaging  modalities.  First, 
registration  of  serial  examinations  can  be  used  to  follow 
regression/progression  of  tumor.  Second,  comparison  of 
registered  MR  images  acquired  before  and  immediately 
after  RF  thermal  ablation  can  be  used  to  determine  whether 
a  tumor  is  adequately  treated.  This  is  particularly  helpful  in 
instances  where  the  edematous  response  to  treatment  can  be 
confused  with  a  highly  perfused  tumor.  Third,  registration 
of  functional,  biochemical  images  such  as  single  photon 
emission  computed  tomography,  positron  emission  tom¬ 
ography  (PET),  and  MR  spectroscopy,  to  anatomical  MR  or 
CT  images  is  useful  for  detecting  and  localizing  cancer  [6] . 
Fourth,  incorporating  the  functional,  biochemical  images 
into  the  iMRI  paradigm  should  aid  image-guided  treatments 
[7].  Finally,  other  treatment  methods  such  as  external 
radiation  therapy,  brachy therapy  [8],  and  surgery,  are  aided 
by  registration  of  images  from  pre-,  intra-,  and  post-therapy 
for  treatment  planning,  guidance,  and  assessment. 

A  few  reports  describe  methods  for  registration  in  the 
pelvis  or  prostate.  Manual  registration  has  been  used  where 
an  operator  cues  on  segmented  vascular  structures  [9]  or 
other  anatomical  landmarks  in  the  pelvis  [6,10-12].  Others 
have  used  automated  3D  schemes  that  match  contours  of 
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bones  and  sometimes  other  structures  that  are  extracted 
using  manual  or  interactive  segmentation  [13-15].  Manual 
segmentation  has  also  been  used  to  create  surfaces  for 
automatic  registration  [16,17].  All  of  these  methods  were 
based  on  rigid  body  registration  and  required  either 
segmentation  or  visual  identification  of  structures. 

We  recently  reported  a  rigid  body  transformation  method 
for  prostate  registration  [18].  For  volume  pairs  acquired 
over  a  short  time  span  from  a  supine  subject  with  legs  flat  on 
the  table,  registration  accuracy  of  both  prostate  centroids 
(typically  <1  mm)  and  bony  landmarks  (average  1.6  mm) 
was  on  the  order  of  a  voxel  (—1.4  mm).  We  obtained 
somewhat  larger  prostate  registration  errors  of  about 
3.0  mm  when  volume  pairs  were  obtained  under  very 
different  conditions,  e.g.  legs  flat  and  legs  raised,  or  with  and 
without  bladder  or  rectal  filling.  Rigid  body  registration  of 
the  pelvis  cannot  follow  prostate  movements  due  to  changes 
in  the  postures  of  legs  and  deformation  of  the  bladder  and 
rectum,  as  reported  by  us  [18]  and  others  [19,20].  In  this 
report,  we  investigate  the  ability  of  warping  registration  to 
express  this  deformation. 

Warping  registration  studies  are  reported  for  the  brain 
[21,22],  for  the  breast  [23-25],  for  a  variety  of  other  organs 
[26-29],  and  for  excised  tissue  [30].  Far  few  reports 
described  results  of  the  pelvis  and  prostate.  Bharaha  et  al. 
recently  reported  a  method  using  manually  segmented 
prostate  for  rigid  body  registration  followed  by  finite 
element-based  warping  in  the  application  of  prostate 
brachytherapy  [31].  Voxel  based  methods,  particularly 
those  based  upon  mutual  information  (MI),  are  robust, 
require  no  segmentation  that  can  be  prone  to  error,  are 
highly  accurate  for  brain  registration  [32],  and  are  suitable 
for  abdominal  registration  where  there  can  be  deformation 
[2].  We  are  investigating  voxel-based  warping  registration 
for  the  particular  application  in  the  pelvis  and  prostate. 

There  are  challenges  to  pelvis  and  prostate  registration. 
First,  pelvic  regions  can  change  shape  significantly,  unlike 
the  brain  to  which  registration  has  been  most  often  applied. 
Different  patient  positions  such  as  legs  flat  and  raised 
significantly  change  the  legs  in  lower  portions  of  image 
volumes  as  well  as  cause  movement  and  deformation  of 
internal  organs  in  the  pelvis.  Second,  the  normal  prostate  is 
a  small  organ  that  when  healthy  measures  only  about  3.8  cm 
in  its  widest  dimension  transversely  across  the  base  [33]. 
Third,  the  small  prostate  is  located  below  a  much  larger 
bladder  that  can  change  shape  and  size.  Fourth,  the  prostate 
might  move  relative  to  the  pelvic  bones  due  to  changes  in 
bladder  and  rectal  filling  [19,20].  The  alignment  of  the 
pelvic  bones,  a  most  prominent  anatomical  feature  in  MR 
gray-scale  images,  does  not  necessarily  mean  that  the 
prostate  is  aligned.  Finally,  efficacious  application  of 
warping  registration  [29,30]  to  interventional  use  requires 
computational  efficiency. 

In  the  present  study,  we  perform  experiments  to  compare 
warping  and  rigid  body  registration  for  the  prostate  and 
pelvis.  By  using  high-resolution  MR  images  giving 


distinctive  anatomic  detail,  we  test  the  ability  of  a  warping 
algorithm  to  correct  anatomical  variations  throughout  the 
pelvic  region.  We  include  conditions  with  very  significant 
changes  in  posture  possible  in  interventional  applications; 
that  is,  we  ttempt  to  register  image  volumes  from  a 
diagnostic  scan  with  legs  flat  to  those  from  a  treatment 
acquisition  with  legs  raised.  We  qualitatively  and  quanti¬ 
tatively  evaluated  registration  results  using  17  volume  pairs 
from  three  volunteers. 


2.  Registration  algorithm 

2.7.  Similarity  measurements 


We  used  two  similarity  measures,  M3  and  correlation 
coefficient  (CC),  in  our  registration.  Suppose  one  volume  R 
is  the  reference ,  and  the  other  F  is  floating .  Their  MI 
MI(R,  F)  is  given  below  [34] 


MI  (R,F)  =  y^pRF(r,f)  log 
rf 


PRF(rJ) 

PR(r>PF(f) 


The  joint  probability  Prf{^J)  and  the  marginal  probabilities 
pR(r)  of  the  reference  image  and  pp(f)  of  the  floating  image, 
can  be  estimated  from  the  normalized  joint  and  marginal 
intensity  histogram,  respectively.  The  CC  CC(R,  F)  is  given 
below  [35]. 


CC(R,F) 


£(fl(r)  -  R(r))(F(f)  -  F(f)) 
Z(R(r)  ~  R(r))2Z.(F(f)  -  F(f  ))2 


Here  R(r ),  F(f )  denote  the  average  intensities  of  the 
reference  and  floating  volumes  and  the  summation  includes 
ail  voxels  within  the  overlap  of  both  volumes. 


2.2.  Rigid  body  registration  algorithm  with  special  features 


Prior  to  warping  registration,  we  perform  rigid  body 
registration  using  a  method  with  features  that  make  it 
particularly  robust  for  MR  pelvic  images.  We  previously 
reported  a  similar  method  [18].  We  use  two  similarity 
measures,  MI  and  CC.  We  use  a  multi-resolution  approach. 
At  low  resolution,  we  resample  both  images  at  1/4  or  1/2 
number  of  voxels  along  each  linear  dimension,  respectively. 
We  use  the  CC  at  these  resolutions  because  it  gives  fewer 
local  maximums  than  MI  [7,18]  and  because  it  can  be 
calculated  faster  than  MI.  We  use  MI  at  full  resolution 
because  the  peaked  similarity  function  gives  a  more  precise 
solution  than  CC  [18].  To  avoid  local  maximums,  we  restart 
with  randomly  perturbed  parameters  obtained  from  a 
uniform  distribution  about  the  initial  transformation  values 
at  the  current  resolution.  The  algorithm  restarts  until  the 
absolute  CC  is  above  an  experimentally  determined 
threshold  or  the  maximum  number  of  restarts  is  reached. 
Absolute  CC  is  used  for  the  restart  test  rather  than  MI 
because  CC  has  a  well-defined  range  between  0  and  1, 
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because  CC  provides  an  independent  check  of  the  MI  result, 
and  because  CC  has  fewer  problems  with  local  and  incorrect 
global  maximums  for  registrations  at  low  resolution  far 
from  the  optimum  value  [18]. 

We  record  all  important  results  following  an  optimiz¬ 
ation  cycle  including  the  CC  and/or  MI  values,  the  number 
of  restarts,  and  the  transformation  parameters.  At  the  end  of 
processing  at  a  lower  resolution,  we  always  select  the 
transformation  parameters  having  the  maximum  CC  value. 
We  then  scale  the  translation  parameters  appropriately  and 
assign  the  new  parameters  to  be  initial  values  at  the  next 
higher  resolution.  At  the  highest  resolution,  we  select  the 
final  transformation  parameters  to  be  those  with  the 
maximum  MI  value. 

Other  details  follow.  A  simplex  algorithm  varies  the  six 
rigid  body  transformation  parameters  (three  translations  and 
three  angles)  to  optimize  the  similarity  measures  [36].  We 
use  an  initial  guess  at  the  lowest  resolution  of  all  zeros 
because  the  patient  is  normally  oriented  approximately  the 
same  way  from  one  scan  to  the  next.  We  set  the  CC 
thresholds  at  a  fixed  value  of  0.50,  and  the  maximum 
numbers  of  restarts  at  10,  5,  and  3,  from  low  to  high- 
resolution,  respectively. 

2.3.  Warping  registration  using  optimized  control  points 

Fig.  1  outlines  the  warping  registration  algorithm  that 
includes  three  major  steps:  control  point  selection,  control 
point  optimization,  and  thin  plate  spline  warping.  The 
unchanging  volume  is  the  reference,  and  the  one  to  be 
warped  is  floating. 


The  manual  selection  of  CP’s  is  an  important  step.  We 
used  RegViz ,  a  program  written  in  Interactive  Data 
Language  (IDL,  Research  System  Inc.,  Boulder,  CO)  and 
created  in  our  laboratory  for  visualizing  and  analyzing 
image  volumes.  Following  rigid  body  registration,  the 
aligned  two  volumes  are  displayed  in  two  rows  slice-by¬ 
slice.  Images  can  be  transverse,  coronal,  or  sagittal  slices.  It 
is  quite  straightforward  to  find  corresponding  features  at  the 
pelvis,  prostate,  bladder,  and  rectum.  We  normally  select 
control  points  (CP’s)  using  recognizable  organ  features  such 
as  comers  and  intersections  of  edges  because  of  their  unique 
positions.  Corresponding  CP’s  in  the  two  volumes  are 
placed  using  a  cursor,  and  sometimes  they  are  in  different 
image  slices.  The  3D  coordinates  are  automatically  stored  in 
a  file.  Because  of  the  optimization  that  occurs  later,  the 
correspondence  can  be  up  to  15  mm  or  « 10  voxels  in  error. 
Experiences  with  CP  selection  are  described  in  Section  4. 
Typically,  we  used  180  CP’s  for  a  volume  with 
256  X  256  X  140  isotropic  voxels. 

The  next  step  of  the  warping  algorithm  (Fig.  1)  is  the  CP 
optimization.  We  define  a  small  cubic  volume  of  interest 
(VOI)  centered  at  each  CP.  The  VOI  can  be  16,  32, 48  or  64 
voxels  on  a  side.  As  reported  later,  the  selection  of  VOI  size 
depends  on  the  amount  of  warping  required.  A  simplex 
optimization  algorithm  varies  the  x,  y,  and  z  transformation 
parameters  of  the  floating  VOI  until  the  MI  with  the 
reference  VOI  is  optimized.  Each  control  point  is  optimized 
independently  and  the  3D  coordinates  of  the  optimal  CP’s 
are  recorded. 

The  final  major  step  is  to  warp  the  floating  volume  using 
the  corresponding  optimal  CP’s  coordinates  to  establish 


Register  two  volumes  using  rigid  body  registration 

Create  a  new  floating  volume  from  rigid  body  registration  parameters 

Select  N  control  points  (CP)  in  the  reference  and  new  floating  volumes 

Record  3D  coordinates  of  the  CP’s  in  both  volumes 

Initialize  the  size  of  volume  of  interest  (VOI)  centered  at  CP’s 

Bin  the  reference  and  floating  volumes  to  256  gray  levels 

FOR  CP  FROM  0  to  N-l  DO  BEGIN 

Optimize  mutual  information  (MI)  between  the  reference  and  floating  VOI’s 

1.  Transform  the  floating  VOI  using  three  translation  parameters 

2.  Interpolate  to  get  a  reformatted  VOI 

3.  Calculate  MI  between  the  reference  and  reformatted  VOI’s 

4.  Vary  the  three  translation  parameters 

5.  Repeat  the  above  steps  1-4  until  meeting  function  tolerance  or  maximum  iteration  number 
Record  the  optimized  CP  coordinates  in  the  floating  volume 

END 

Calculate  thin  plate  spline  transformation  using  the  reference  and  optimized  CP’s 
Interpolate  the  floating  volume  and  get  a  warped  volume 

Fig.  1.  Flow  chart  of  the  warping  registration  algorithm.  Following  rigid  body  registration,  N  CP’s  are  selected  in  both  the  reference  and  floating  volumes.  A 
small  cubic  VOI  is  centered  on  each  control  point.  Optimization  is  performed  by  varying  the  x,  y,  and  z  locations  of  the  floating  VOI  until  the  MI  between 
corresponding  voxels  is  maximized.  Each  control  point  is  optimized  independently,  and  then  the  optimized  CP’s  are  used  to  establish  a  three-dimensional  thin 
plate  spline  transformation  for  the  entire  volume. 
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a  3D  thin-plate  spline  (TPS)  transformation  [37,38].  We 
now  briefly  go  through  the  three  computing  steps  for  the 
TPS  transformation. 

First,  let  Pi  =  (xi.J’lZ,).  P2  =  (x2,y2,z2),...,  P„  = 
(xn^ymZ„)  be  n  control  points  in  the  image  coordinate  of 
the  reference  volume.  Write  rtJ  =  P,  -  !  for  the  distance 

between  point  i  and  j.  We  define  matrices 


1  x{  yi  z, 

i  *2  y2  z2 


n  X4; 


L  1  xn  yn  zn  J 
r  0  rl2  r  13  •••  r,„ 

I  r2l  0  r23  ••  r1n 


nXn; 


Lrn\  rn  2  rn3  •••  0 

and 


‘  K  P~ 

L=  T  ,  (n  +  4)  X  («  +  4); 
_PJ  0_ 


where  T  is  the  matrix  transpose  operator  and  O  is  a  4  X  4 
matrix  of  zero. 

Second,  let  Q{  =  (iq,  vl5  Wj),  <22  ^ 

(w2,v2,w'2),...,(3w  =  be  n  corresponding  CP's 

in  the  image  coordinate  of  the  floating  volume.  We  get 


3  Xn, 


matrices 

"«1 

u2  • 

ut 

Vl 

v2  • 

*•  v. 

_W\ 

w2  * 

**  w, 

o 

'w' 

II 

0  0 

0)T, 

3x(«  +  4), 


and  define  the  vector  W  =  (wj,>v2,..., wn)  and  the  coeffi¬ 
cients  ax ,  «*,  a^,  and  az  by  the  equation 

L~lY  =  (W\ax  au  av  aw)T . 

Third,  use  the  elements  of  LTlY  to  define  a  function 
f{u\^,W)  everywhere  in  the  entire  volume 


n 

/(i/,v',w/)  =  ax  +  auu  +  avv  +  aww  +  ^Wi\Pi  -  (w,v,w)l. 

i=0 


Thus  any  voxel  (uhvhWi)  in  the  floating  volume  is 
transformed  to  a  new  coordinate  (w-,v-,w-)  and  a  warped 
volume  can  be  obtained  by  trilinear  interpolation. 

Additional  algorithm  details  are  now  described.  For  both 
VOI  optimization  and  rigid  body  registration,  we  use 
trilinear  interpolation.  Optimization  of  similarity  ends  either 
when  the  maximum  number  of  calculations  is  reached 
(typically  500)  or  the  fractional  change  in  the  similarity 
function  is  smaller  than  a  tolerance  (typically  0.001).  We 
use  IDL  as  the  programming  language. 


3.  Experimental  methods 

3.1.  Image  acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens 
MRI  system  (Magnetom  Symphony,  Siemens  Medical 
Systems,  Erlangen,  Germany).  An  8-element  phased  array 
body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a 
uniform  sensitivity.  Typically  two  anterior  and  two 
posterior  elements  were  enabled  for  signal  acquisition.  We 
used  two  different  MR  sequences.  First,  a  3D  FLASH 
sequence  with  TR/TE/flip  parameters  of  12/5.0/60  gave 
256  X  256  X  128  voxels  over  a  330  X  330  X  256  mm3  field 
of  view  (FOV)  to  yield  1.29  X  1.29  X  2.0  mm3  voxels 
oriented  to  give  the  highest  resolution  for  transverse 
slices.  This  sequence  was  good  for  pelvic  imaging  but 
was  not  ideal  for  prostate  visualization  and  it  was  used  for 
volunteer  SI.  Second,  a  3D  PSIF  sequence  with 
9.4/5.0/60  (TR/TE/flip)  yielded  160  X  256  X  128  voxels 
over  a  219  X  350  X  192  mm3  rectangular  FOV  and 
1.4  X  1.4  X  1.5  mm3  voxels  oriented  to  give  the  highest 
resolution  for  transverse  slices.  There  was  over  sampling  at 
31%  in  the  slice  direction  to  reduce  aliasing  artifacts.  The 
second  sequence  gave  excellent  image  contrast  for  the 
prostate  and  its  surroundings  and  it  was  used  for  volunteers 
S2  and  S3. 

3.2.  Imaging  experiments 

We  acquired  3D  MRI  volume  images  from  three 
normal  volunteers  under  a  variety  of  conditions  simulat¬ 
ing  anticipated  conditions  in  diagnostic  and  treatment 
applications.  Before  image  acquisition,  each  volunteer 
drank  water  and  had  a  relatively  full  bladder.  In  the 
diagnostic  position ,  the  subject  laid  supine  throughout 
MR  scanning.  In  the  treatment  position ,  the  subject  was 
supine,  and  his  legs  were  supported  at  30-60°  relative  to 
the  horizon  and  separated  in  a  ‘V’  with  an  angle  of  60- 
90°  between  two  legs.  This  is  similar  to  the  lithotomy 
position  used  in  prostate  therapies,  and  it  should  provide 
access  for  needle  insertion  in  brachytherapy  or  RF 
thermal  ablation.  In  some  experiments,  the  subject 
micturated  to  create  an  empty  bladder  prior  to  imaging. 
For  each  subject,  image  volumes  were  typically  obtained 
on  the  same  day  within  a  2  h  session.  We  imaged  one 
volunteer  (S3)  a  week  before  the  standard  imaging 
session,  and  we  refer  to  these  volumes  as  diagnosis  1 
week.  Between  volume  acquisitions,  volunteers  got  off 
the  MRI  table,  stretched,  and  walked  around  to  ensure 
that  they  would  assume  a  different  position  when  they 
laid  back  on  the  table.  The  coil  array  was  centered  on 
the  prostate.  All  images  of  a  volunteer  were  acquired 
with  the  same  MRI  acquisition  parameters  so  as  to 
ensure  very  similar  gray  values.  In  total,  there  are  4,  4, 
and  8  volumes  for  volunteer  SI,  S2,  and  S3,  respectively. 
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The  permutation  of  the  volumes  gives  many  possible 
volume  pairs  for  registration  experiments. 

3.3.  Volumes  for  registration  experiments 

We  registered  17  volume  pairs  under  five  different 
conditions  as  defined  above.  Five  pairs  are  treatment- 
diagnosis: ;  seven  pairs  are/«//  bladder -empty  bladder,  two 
pairs  are  diagnosis  1  week-diagnosis',  and  three  pairs  are 
diagnosis -diagnosis.  For  each  case,  other  conditions  were 
controlled.  For  example,  for  the  case  of  diagnosis  1  week- 
diagnosis,  both  volumes  were  acquired  with  empty  bladder 
and  comparable  conditions.  Rigid  body  and  warping 
registration  were  applied  to  each  of  the  volume  pairs. 
Results  were  evaluated  as  described  next. 

3.4.  Registration  evaluation 

We  used  the  multiple  visualization  features  of  RegViz  to 
visually  evaluate  registration  results.  First,  we  manually 
segmented  prostate  boundaries  in  image  slices  and  copied 
them  to  corresponding  slices  from  the  other  volume.  This 
enabled  visual  determination  of  the  overlap  of  prostate 
boundaries  over  the  entire  volume.  We  applied  the  same 
method  to  evaluate  pelvic  registration.  Second,  color  overlay 
displays  were  used  to  evaluate  overlap  of  structures.  One 
image  was  rendered  in  gray  and  the  other  in  the  ‘hot-iron’ 
color  scheme  available  in  IDL.  To  visualize  potential 
differences,  it  was  quite  useful  to  interactively  change  the 
contribution  of  each  image  using  the  transparency  scale. 
Third,  we  used  a  sector  display,  which  divided  the  reference 
and  registered  images  into  rectangular  sectors  and  created  an 
output  image  by  alternating  sectors  from  the  two  input 
images.  Even  subtle  shifts  of  edges  could  be  clearly  seen  [18]. 

Voxel  gray  value  measures  were  calculated  as  indicators 
of  registration  quality.  MI  and  CC  between  registered 
volumes  were  computed.  Since  volumes  to  be  registered 
were  acquired  using  the  same  acquisition  parameters,  high 
absolute  CC  values  were  obtained  when  registration  was 
good  [24].  Because  voxel  intensities  were  comparable,  we 
created  difference  images  and  calculated  statistics  such  as 
the  voxel  mean  and  standard  deviation  following 
registration. 

Finally,  we  used  a  variety  of  tools  in  RegViz  to  evaluate 
registration  quality.  We  used  contour  overlap  and  color 
overlay  to  assess  the  prostate  registration.  We  manually 
segmented  the  prostate  across  all  slices  and  calculated  the 
potential  displacements  of  the  prostate  3D  centroid. 

4,  Results 

4.1.  Effect  of  control  point  selection  on  registration  quality 

In  well  over  100  registration  experiments  using 
different  numbers  and  placement  of  CP’s,  we  investigated 


effects  on  warping  registration  quality.  For  each  of  the 
three  volunteers,  we  selected  one  typical  volume  pair 
from  the  diagnostic-treatment  positions  for  systematic 
experiments.  We  progressively  increased  the  number  of 
CP’s  from  15  to  250.  We  found  that  less  than  120  CP’s 
did  not  produce  good  visual  matching  of  our  high- 
resolution  MR  images  showing  great  anatomical  detail. 
More  than  220  CP’s  did  not  give  significant  improve 
results  but  required  more  time  for  manual  selection  and 
optimization.  When  we  used  «180  CP’s  placed  strate¬ 
gically  using  rules  described  later,  we  obtained  excellent 
results  over  the  entire  pelvis  and  internal  organs.  As  a 
result  of  our  experience,  we  modified  the  registration 
method  to  be  suitable  for  many  CP’s  (Section  5). 

Some  rules  follow  for  strategic  placement  of  CP’s.  For 
registration  of  treatment  and  diagnostic  image  volumes,  most 
CP’s  were  selected  using  transverse  slices  because  they  best 
showed  the  pelvic  displacement  when  moving  the  legs  to  the 
treatment  position  (Fig.  2).  About  25  CP  pairs  were  placed 
near  edge  and  point  features  having  recognizable  correspon¬ 
dence  on  each  of  5-8  transverse  slices  with  a  z  interval  of 
~  8  mm,  covering  the  entire  pelvic  region.  Additionally,  we 
placed  about  25  CP’s  from  sagittal  slices  because  they 


Fig.  2.  Control  point  selection  when  images  are  acquired  in  the  treatment 
and  diagnostic  positions.  Image  (a)  is  from  the  reference  volume  acquired 
in  the  treatment  position  with  legs  raised.  Image  (b)  is  to  be  warped  and  is 
from  the  volume  acquired  in  the  diagnostic  position  with  the  subject  supine 
on  the  table.  Transverse  slices  best  show  the  deformations,  especially  at  the 
legs.  As  described  in  the  text,  CP’s  indicated  by  the  white  dots  are  selected 
around  the  pelvic  surface  and  the  prostate.  Each  control  point  is  located  at 
one  voxel  but  displayed  much  bigger  for  better  visualization.  Volumes  are 
from  volunteer  S2. 
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provided  other  structures  that  can  be  missed  in  the  transverse 
images.  It  was  also  important  to  include  CP’s  from  organs 
other  than  the  prostate  because  they  constrained  warps.  We 
always  placed  CP’s  at  critical  regions  such  as  the  prostate 
center,  pelvic  surface,  bladder  border,  and  rectal  walls. 

For  registration  of  image  volumes  with  full  and  empty 
bladder,  most  CP’s  were  placed  from  sagittal  slices 
because  they  best  showed  the  deformation  of  the  bladder 
and  rectum  (Fig.  3).  About  10-20  CP’s  were  placed  at 
the  borders  of  the  bladder  and  rectum  on  each  of  8-10 
sagittal  slices  with  an  equal  interval  of  «  8  mm,  covering 
the  entire  pelvic  region  including  the  prostate,  bladder, 
and  rectum. 

4.2.  Registration  quality  of  warping  and  rigid  body 
registration 

In  Fig.  4,  we  compare  warping  and  rigid  body 
registration  for  a  typical  volume  pair  in  the  treatment  and 
diagnostic  positions.  Following  warping  registration,  the 
prostate  boundary  overlap  is  excellent  (Fig.  4(e))  and 
probably  within  the  manual  segmentation  error.  Similar 
results  were  obtained  in  other  transverse  slices  throughout 
the  prostate.  The  prostate  3D  centroid  calculated  from 
segmented  images  displaced  by  only  0.6  mm,  or  0.4  voxels, 
following  warping.  Following  rigid  body  registration,  the 
prostate  was  misaligned  with  a  displacement  to  the  posterior 
of  *®3.4  mm  when  in  the  treatment  position  (Fig.  4(d)),  as 
previously  reported  by  us  [18].  Using  rigid  body  regis¬ 
tration,  there  is  significant  misalignment  throughout  large 
regions  in  the  pelvis  (Fig.  4(f))  that  is  greatly  reduced  with 
warping  (Fig.  4(g)).  Note  that  warping  even  allows  the  outer 
surfaces  to  match  well.  Other  visualization  methods  such  as 


Fig.  3.  Control  point  selection  when  images  are  acquired  with  a  week 
interval  between  them.  Image  (a)  is  from  the  reference  volume  acquired  1 
week  later  with  an  empty  bladder.  Image  (b)  is  to  be  warped  and  is  from  the 
volume  acquired  earlier  with  a  full  bladder.  Sagittal  slices  best  show  the 
deformations  at  the  bladder  (vertical  arrow)  and  rectum  (horizontal  arrow) 
where  most  CP’s  are  placed.  Volumes  are  from  volunteer  S3. 


two-color  overlays  and  difference  images,  quickly  show 
matching  of  structures  without  segmentation  but  do  not 
reproduce  well  on  a  printed  page. 

We  next  examine  the  effect  of  conditions  such  as 
bladder  and  rectal  filling  that  might  change  from  one 
imaging  session  to  the  next.  In  Fig.  5,  we  compare 
warping  and  rigid  body  registration  for  a  volume  pair 
with  1  week  between  imaging  sessions.  One  volume  is 
with  an  empty  bladder  and  the  other  is  with  a  relatively 
full  bladder.  There  is  also  a  difference  in  rectal  filling. 
Warping  registration  closely  aligns  the  prostate  Fig.  5(e)) 
while  rigid  body  does  not  (Fig.  5(d)).  In  addition,  rigid 
body  registration  does  not  align  the  bladder  and  parts  of 
the  rectum  (Fig.  5(f)).  With  warping,  the  bladder  closely 
matches  the  reference,  and  the  rectum  is  better  aligned 
(Fig.  5(g)).  Other  visualization  methods  showed  excellent 
alignment  of  internal  and  surface  edges.  Difference 
images  show  that  warping  greatly  improves  alignment 
of  internal  structures  as  compared  to  rigid  body 
registration  (Fig.  6).  The  difference  image  following 
rigid  body  registration  shows  bright  regions  indicating 
misalignments  (Fig.  6(d))  that  are  removed  with  warping 
(Fig.  6(e)). 

We  also  examined  volume  pairs  with  both  volumes 
acquired  in  the  diagnostic  position  under  comparable 
conditions.  In  the  current  data  set,  five  volume  pairs  fit 
these  criteria.  In  all  such  cases,  rigid  body  registration 
worked  as  well  as  warping.  There  were  no  noticeable 
deformations  in  the  pelvis,  and  prostate  centroids  typically 
displaced  less  than  1.0  mm  between  the  two  registered 
volumes.  Note  that  this  was  obtained  even  though  subjects 
always  got  up  from  the  table  and  moved  around  before  being 
imaged  again. 

4.3.  Quantitative  evaluation  of  warping  registration 

Fig.  7  shows  the  CC  and  MI  values  between  registered 
volumes.  Warping  increased  CC  and  MI  values  in  every 
case,  and  a  paired  two-tailed  t  test  indicated  a  significant 
effect  of  warping  at  p  <  0.5%.  The  most  significant 
improvement  was  in  the  case  of  treatment-diagnosis  where 
improvements  in  CC  and  MI  were  as  high  as  102.7  and 
87.8%,  respectively. 

Statistics  of  image  differences  following  rigid  body  and 
warping  registration  are  shown  in  Fig.  8.  Warping  reduces 
the  absolute  intensity  difference  between  corresponding 
voxels  (Fig.  8(a)),  and  the  mean  across  all  image  volumes  is 
only  4.2  gray  levels,  a  value  corresponding  to  only  4.7%  of 
the  mean  image  value  of  90.  We  used  the  absolute  intensity 
difference  because  signed  values  canceled  when  averaged 
over  the  entire  image.  The  standard  deviation  of  absolute 
difference  is  also  reduced  (Fig.  8(b)). 

These  quantitative  measures  match  observation  from 
visual  inspection.  For  example,  the  third  pair  of  the  first 
group  (diagnosis-treatment)  in  Fig.  7  and  8  corresponds  the 
images  in  Fig.  4.  After  warping,  registration  greatly 
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Fig.  4.  Comparison  of  warping  and  rigid  body  registration  for  volumes  acquired  in  the  treatment  and  diagnostic  positions.  Image  (a)  is  from  the  reference 
volume  acquired  in  the  treatment  position,  and  the  prostate  is  manually  segmented.  Images  in  the  left  and  right  columns  are  from  the  floating  volume  acquired 
in  the  diagnostic  position  following  rigid  body  and  warping  registration,  respectively.  To  show  potential  mismatch,  the  prostate  contour  from  the  reference  in 
(a)  is  copied  to  (b)  and  (c)  and  magnified  as  the  dashed  contours  in  (d)  and  (e).  The  3  mm  movement  of  the  prostate  to  the  posterior  is  corrected  with  warping  (e) 
but  not  rigid  body  registration  (d).  Pelvic  boundaries  manually  segmented  from  the  reference  show  significant  misalignment  with  rigid  body  (f)  that  is  greatly 
improved  with  warping  (g).  Images  are  transverse  slices  from  volunteer  S2. 


improved.  Another  interesting  example  is  the  difference 
images  in  Fig.  6(d)  and  (e)  that  correspond  to  the  last  pair  of 
the  second  group  (full-empty  bladder)  in  Fig.  8.  Once 
again,  the  statistical  measures  reflect  the  great  change  in 
visual  quality. 


4.4.  Algorithmic  implementation 

In  rigid  body  registration,  the  multi-resolution 
approach  and  restarting  algorithm  were  important  modi¬ 
fications.  First,  these  two  features  improved  robustness. 
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Fig.  5.  Comparison  of  rigid  body  and  warping  registration  for  volumes  acquired  with  an  interval  of  1  week  between  imaging  sessions.  The  reference  image  (a) 
with  a  manually  segmented  prostate  was  acquired  later  with  an  empty  bladder  (vertical  arrow)  and  partial  rectal  filling  (horizontal  arrow).  Images  in  the  left  and 
right  columns  are  from  the  floating  volume  acquired  earlier  following  rigid  body  and  warping  registration,  respectively.  To  show  potential  mismatch,  contours 
from  the  reference  are  shown  on  images  following  registration,  as  described  in  Fig.  4.  The  full  bladder  in  (d)  has  pushed  the  prostate,  shown  by  the  continuous 
curve,  in  the  caudal  direction.  After  warping,  prostate  contours  match  closely  (e).  The  bladder,  rectum,  and  other  organs  closely  align  following  warping  (g). 
With  rigid  body  (f),  proceeding  from  left  to  right,  the  front  of  the  pelvis,  the  bladder  (arrow),  and  the  rectum  are  all  misaligned.  Images  are  sagittal  slices  from 
volunteer  S3. 


The  algorithm  always  gave  very  nearly  the  same 
transformation  parameters  (<0.01  voxels  and  0.01 
degrees)  for  the  17  volume  pairs  in  this  study  using  a 
wide  variety  of  initial  guesses.  We  also  found  that  MI 
was  more  accurate  than  CC  at  the  highest  resolution 
[18].  Second,  the  multi-resolution  approach  enabled  the 
program  to  get  close  to  the  final  value  quickly  because  of 
the  reduced  number  of  calculations.  That  is,  the  time  for 
reformatting  at  the  lowest  resolution  of  1/4  number  of 
voxels  in  a  linear  dimension  was  0.16  min,  less  than  1/63 
times  that  at  the  highest  resolution,  a  value  nearly  equal 
to  the  1/64  expected  from  the  change  in  the  number  of 
voxels.  In  a  typical  example,  the  number  of  restarts  was 
5,  1,  and  1  for  resolutions  at  1/4,  1/2,  and  the  full 
number  of  voxels  in  a  linear  dimension,  respectively. 
When  we  checked  the  restarts  at  the  resolution  of  1/4 
number  of  voxels,  we  determined  that  none  of  the  five 
restarts  converged  to  the  same  transformation.  It  has  been 


our  experience  that  more  restarts  are  desirable  at  the 
lower  resolutions,  and  the  algorithm  includes  this  feature. 
Each  call  to  the  simplex  optimization  resulted  in  50-100 
MI  evaluations  before  the  tolerance  (0.001)  was  reached. 
In  some  experiments  on  multiple  volumes,  we  reduced 
the  tolerance  value  but  found  little  difference  in 
registration  quality,  probably  because  of  the  restarting 
and  multi-resolution  features.  The  time  for  rigid  body 
registration,  typically  5-10  min  on  a  Pentium  IV, 
1.8  GHz  CPU,  with  1.0  GB  of  memory,  could  possibly 
be  reduced  to  within  1  min  with  optimized  C  code  rather 
than  the  high  level  language  IDL. 

Some  technical  aspects  of  warping  registration  are  of 
interest.  Fig.  9  shows  the  optimization  time  and  MI  values 
between  registered  volumes  as  a  function  of  VOI  size.  The 
optimization  time  for  180  CP’s  increases  roughly  linearly 
with  the  number  of  voxels  within  a  VOI,  about  0.5  min  for 
VOI’s  with  16  voxels  on  one  side  and  30  min  for  VOI’s  with 
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Fig.  6.  Comparison  of  registration  quality  for  rigid  body  and  warping  registration.  The  reference  image  (a)  was  acquired  with  a  relatively  empty  bladder 
(arrow).  Images  (b)  and  (c)  are  from  the  floating  volume  acquired  with  a  full  bladder  following  rigid  body  and  warping  registration,  respectively.  Images  (d) 
and  (e)  are  the  absolute  difference  images  between  the  reference  and  registered  images,  respectively.  Bright  regions  following  rigid  body  indicate 
misalignments  (d)  that  are  removed  with  warping  (e).  Images  (d)  and  (e)  are  displayed  using  the  same  gray-scale  window  and  level  values.  Images  are  coronal 
slices  from  S3  volumes  shown  in  Fig.  5. 


64  voxels  on  a  side.  In  Fig.  9,  the  MI  curve  saturates  at  the 
VOI  size  of  64  voxels  on  a  side  that  means  the  size  of  64 
gave  better  MI  value.  These  curves  are  for  the  case  of 
treatment-diagnosis  for  volunteer  S2.  When  we  examined 
the  cases  of  full -empty  bladder  and  volumes  acquired  over 
1  week  time  interval,  we  found  that  the  VOI  size  of  16 
voxels  on  a  side  worked  best.  Using  the  same  computer 
above,  for  a  volume  with  256  X  256  X  140  voxels  and  180 
CP’s,  the  warping  registration  typically  takes  about  15- 
45  min  depending  on  the  VOI  size. 

We  report  some  details  on  VOI  optimization  for  a  typical 
treatment-diagnosis  volume  pair  from  subject  S2.  Following 
rigid  body  registration,  the  mean  distance  between  the 
manually  selected  reference  and  floating  CP’s  was 
15.5  ±  10.7  mm,  where  the  latter  number  is  the  standard 
deviation.  The  maximum  distance  was  53.2  mm.  After  VOI 
optimization,  the  algorithm  moved  the  floating  CP’s  an 
average  of  9.0  ±  6.5  mm.  This  value  shows  that  one  does 
not  have  to  be  very  careful  in  marking  corresponding  CP’s. 


5.  Discussions  and  conclusions 

5.7.  Applicability  of  warping  registration 

For  MR  images  of  the  pelvis  and  prostate,  warping 
registration  is  desirable  whenever  images  are  acquired  in 
different  positions  or  with  different  conditions  of  bladder 
and  rectal  filling.  Local  deformations  throughout  the  pelvis 
can  be  corrected,  and,  more  importantly,  the  prostate  can  be 
accurately  registered.  However,  when  images  are  acquired 


in  the  same  position  under  comparable  conditions  such  as 
our  case  called  diagnosis -diagnosis,  rigid  body  registration 
worked  satisfactorily  as  previously  reported  by  us  [18]. 
Similarly,  if  one  were  to  reproduce  the  treatment  position 
with  reasonable  accuracy,  we  believe  that  prostate  regis¬ 
tration  would  be  very  good. 

Our  goal  is  to  get  good  matching  throughout  the  entire 
pelvic  region  not  just  at  the  prostate  because  proper 
localization  of  other  organs  is  important  for  interpretation 
of  some  functional  images  and  because  anatomical  spatial 
integrity  is  important  for  treatment  planning.  Hence,  we 
used  high-resolution  MR  images  that  provide  a  very 
stringent  test  for  warping.  Many  anatomical  details  are 
evident,  and  even  a  small  mismatch  can  clearly  be  seen.  As  a 
result,  we  found  that  —180  CP’s  were  required  to  get 
excellent  quality  registration.  When  we  applied  the  method 
to  register  CT  images  with  PET  images  of  the  lung  having 
much  less  resolution,  many  fewer  points  («50)  were 
required  [39].  With  a  sufficient  number  of  CP’s,  the  TPS 
transformation  excellently  approximated  the  deformations 
of  the  pelvis  and  internal  structures  of  our  MR  images.  Even 
when  we  warped  the  volume  in  the  diagnostic  position  to 
one  in  the  treatment  position,  most  organs  were  closely 
aligned,  despite  very  significant  movements.  The  method 
performed  equally  well  for  correcting  the  deformation  and 
organ  displacement  arising  from  changes  in  bladder  and 
rectal  filling. 

With  our  graphical  user  interface,  interactive  control 
point  selection  is  quite  easy  after  training.  It  usually  took  an 
experienced  user  about  15  min  to  select  180  CP’s.  Based  on 
our  experience,  we  think  that  it  is  possible  to  create  an 
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Diagnosis  ~  Full  Bladder  ~  Diagnosis  Diagnosis  ~ 

Treatment  Empty  Bladder  ~  Diagnosis  Diagnosis 

1  Week 

Fig.  7.  Voxel  similarity  measures  for  rigid  body  and  warping  registration.  CC  (a)  and  MI  (b)  following  registration  with  (light  bars)  and  without  (dark  bars) 
warping  are  plotted.  Conditions  described  in  Section  3  are  listed  on  the  *-axis.  Warping  increased  CC  and  MI  in  all  cases.  The  most  significant  increases 
occurred  in  the  case  of  the  treatment-diagnosis  volume  pairs  where  maximum  increases  in  CC  and  MI  are  102.7%  and  87.8%,  respectively.  For  volumes 
acquired  with  in  the  same  diagnostic  position  and  comparable  conditions  (two  right  most  groups),  warping  did  not  have  significant  improvement  over  rigid 
body  method. 
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Fig.  8.  Image  statistics  of  absolute  intensity  difference  images  for  rigid  body  and  warping  registration.  The  mean  (a)  and  standard  deviation  (b)  are  plotted.  See 
the  legend  of  Fig,  7  for  other  details.  Warping  decreased  the  mean  and  standard  deviation  in  each  case,  but  the  most  significant  decreases  occurred  in  the  case 
of  the  treatment-diagnosis  volume  pairs.  After  warping,  the  intensity  averaged  over  all  data  is  4.2  ±1.9  gray  levels,  a  value  corresponding  to  only  ^4.7%  of 
the  mean  image  value  of  ~  90  gray  levels. 
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VOI  Size  (Voxel  on  a  side) 

Fig.  9.  Optimization  time  and  MI  as  a  function  of  VOI  size.  The  left  vertical 
axis  is  MI,  and  the  right  vertical  axis  is  the  total  VOI’s  optimization  time. 
The  horizontal  axis  is  the  size  of  the  VOI  on  a  side.  In  each  case,  the  VOI  is 
centered  on  the  CP,  but  since  even  numbers  of  voxels  are  used,  the  CP  is 
displaced  consistently  to  the  upper  left  hand  comer  by  one  voxel.  With 
increasing  VOI  size,  time  increases  linearly  with  the  number  of  voxels 
within  the  VOI.  The  peak  MI  value  is  at  a  VOI  size  of  64  on  a  side.  A 
treatment-diagnosis  volume  pair  is  used  from  S2  with  180  CP’s. 

automatic  or  semiautomatic  method  for  selection  of 
appropriate  CP’s  in  the  pelvis.  For  example,  one  might 
use  a  gray  scale  threshold  to  detect  the  pelvic  outer 
boundaries  and  apply  edge  enhancement  to  extract  feature 
of  internal  structures.  CP’s  would  be  placed  on  such 
structures  automatically.  We  are  investigating  this  and 
other  methods  for  CP  selection  followed  by  automatic 
warping  registration. 

One  way  to  adjust  the  movement  of  CP’s  is  to  change  the 
size  of  the  VOL  In  the  case  of  treatment-diagnosis  volume 
pairs,  a  large  VOI  size  of  64  on  a  side  worked  better  than 
smaller  ones  because  displacements  were  large,  because 
larger  VOI’s  tend  to  give  a  more  robust  optimization,  and 
because  no  small  local  deformations  were  required. 
However,  a  size  of  16  on  a  side  worked  better  for  the  case 
of  full-empty  bladder  volume  pairs  because  small  VOI’s 
better  capture  the  small,  local  deformations.  VOI’s  with  a 
size  of  64  on  a  side  covered  most  of  the  bladder  and  could 
not  generate  small  local  deformations.  For  volumes  with 
both  large  and  small-scale  deformations,  we  suggest  using 
different  VOI  sizes  for  different  CP’s. 

With  warping  registration,  we  have  to  be  concerned 
about  potential  warping  errors  affecting  the  application  of 
interest.  For  the  prostate,  we  used  3-5  CP’s  near  the 
prostate  center  because  we  desired  to  maintain  the  spatial 
integrity  of  the  organ  and  to  preserve  the  tissue  volume.  We 
placed  many  CP’s  around  the  pelvic  surface  to  produce 
reasonable  warping. 

5.2.  Evaluation  of  warping  registration 

Since  there  is  no  gold  standard  for  warping  registration  of 
anatomical  images,  we  used  a  variety  of  methods  to  evaluate 


registration  quality.  First,  for  routine  evaluation,  a  color 
overlay  is  simple,  fast,  and  intuitive.  To  better  visualize  the 
two  data  sets,  we  interactively  adjust  the  transparency  scale 
of  each  image.  Second,  for  illustration  of  subtle  difference 
along  an  edge,  we  recommend  a  sector  display  because  it 
best  shows  small  shifts.  Third,  for  visual  evaluation  of  a 
specific  organ  such  as  the  prostate,  we  like  to  superimpose 
manually  marked  contours  from  one  image  onto  another  as 
shown  in  Figs.  4  and  5.  This  clearly  shows  any  displacement 
or  deformation  even  in  a  printed  figure.  Fourth,  a  more 
quantitative  approach  is  obtained  by  calculating  the 
displacement  in  millimeters  from  the  3D  centroid  of  a 
segmented  organ  such  as  the  prostate.  Finally,  when  images 
have  comparable  gray  levels,  a  difference  image  can  provide 
a  visual  evaluation  or  a  quantitative  evaluation  from  image 
statistics.  A  downside  with  MR  difference  images  is  that  the 
inhomogeneity  of  the  signal  response  and  interpolation  can 
introduce  artifacts  in  difference  images.  Since  MR  image 
intensity  can  vary  with  different  MR  sequence  parameters 
and  the  signal  response  of  MR  coil,  gray  value  statistic  may 
have  some  limitations  when  image  acquisitions  are  not 
carefully  repeated. 

5.3.  Algorithmic  robustness  and  efficiency 

The  rigid  body  algorithm  is  robust  for  a  global 
registration.  Because  of  two  principal  design  features,  the 
algorithm  is  quite  robust  and  accurate  for  volume  pairs 
acquired  in  the  same  positions  and  with  comparable 
conditions  [18].  First,  using  both  CC  and  MI  at  different 
resolutions  was  an  important  feature  that  increased  robust¬ 
ness.  CC  gave  fewer  local  minimums  at  low  resolutions  and 
MI  was  more  accurate  at  high-resolution  [7,18].  Second,  the 
restarting  mechanism  was  also  quite  important.  Without 
restarting,  we  found  that  registrations  sometimes  failed  in 
cases  of  volumes  with  large  mismatches  and  significant 
deformation.  Even  these  cases  resulted  in  a  proper  solution 
when  restarting  was  employed. 

Based  upon  our  initial  experiments  with  interactive  CP 
selection,  we  determined  that  many  CP’s  were  required  for 
good  matching  throughout  the  pelvis.  As  a  result,  we 
designed  algorithm  features  to  be  computationally  efficient 
for  TPS  warping  with  hundreds  of  CP’s.  First,  the 
optimization  of  small  VOI’s  is  very  fast.  Second,  we 
optimized  each  CP  separately  because  the  optimization  of 
three  parameters  (x,  y ,  and  z )  is  simple  and  fast.  Conversely, 
as  previously  reported  by  others  [26,27],  the  simultaneous 
optimization  of  many  CP’s  leads  to  a  much  more  complicated 
error  surface  and  local  maximums.  If  one  were  to  use  180 
CP’s  and  optimizes  the  540  free  parameters  simultaneously, 
the  optimization  process  would  become  extraordinarily 
complex.  Third,  we  applied  the  TPS  transformation  once  to 
the  final,  optimal  CP’s;  this  saved  considerable  time.  If  TPS 
was  applied  in  each  iteration,  the  registration  time  would  be 
unacceptable  for  our  application.  If  we  were  to  use  optimized 
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C  code,  the  total  time  for  rigid  body  and  warping  registration 
should  reduce  to  within  5  min. 

5.4.  Applications 

We  discuss  several  points  on  volume  interpolation.  We 
used  trilinear  interpolation  in  the  algorithm  because  it  is 
fast.  The  final  floating  volume  following  rigid  body 
registration  is  important  because  it  is  used  for  CP  selection 
and  warping  registration.  A  high  accurate  interpolation  such 
as  sine  spline  [40]  can  be  applied  to  obtain  this  volume  with 
reduced  interpolation  errors.  For  optimization,  partial 
volume  interpolation  that  was  reported  robust  for  MI- 
based  registration  [34]  is  another  option  for  improvement. 

The  flexibility  introduced  with  manual  selection  of  CP’s 
makes  the  current  software  suitable  for  warping  registration 
in  many  applications  in  addition  to  the  clinical  procedures 
described  in  Section  1.  We  have  successfully  applied  it  to 
human  MR-MR  prostate  images  as  shown  here,  rat  CT -CT 
images,  and  CT-PET  lung  images  [39].  We  believe  that  the 
registration  method  can  be  applied  to  many  organs  other 
than  the  pelvis  and  prostate,  multi-modality  images,  and 
inter-subject  images.  In  addition,  we  think  it  applicable  to  a 
variety  of  animal  experiments  in  which  we  are  involved, 
including  iMRI-guided  thermal  ablation  in  pig  and  rabbit, 
prostate  imaging  studies  in  dog,  and  controlled  drug  release 
studies  in  rat. 

We  conclude  that  our  MI  warping  registration  is  fast  and 
can  be  applied  to  a  variety  of  applications.  For  prostate  and 
pelvic  imaging,  it  works  better  than  rigid  body  registration 
whenever  the  subject  position  or  condition  is  greatly 
changed  between  acquisitions.  It  will  probably  be  a  useful 
tool  for  many  applications  in  prostate  diagnosis,  staging, 
and  therapy. 

6.  Summary 

Many  applications  in  prostate  cancer  management  such 
as  tumor  localization,  possibly  tumor  staging,  tumor 
targeting  during  therapy,  assessment  of  adequate  treatment, 
and  treatment  follow  up,  require  image  registration  of  MRI 
volumes  and/or  volumes  from  other  imaging  modalities. 
With  regard  to  interventional  MRI  guided  RF  thermal 
ablation  for  the  minimally  invasive  treatment  of  prostate 
cancer,  registration  applications  include  the  comparison  of 
registered  MR  images  acquired  before  and  immediately 
after  RF  ablation  to  determine  whether  a  tumor  is 
adequately  treated.  When  images  are  acquired  in  different 
patient  positions  and/or  different  conditions,  the  pelvis, 
prostate,  bladder,  and  rectum  can  deform  and  displace. 
Warping  registration  is  desired  to  correct  for  such 
deformations. 

We  created  a  two-step,  3D  registration  algorithm  using 
MI  and  thin  plate  spline  warping  for  the  prostate  MR 
images.  First,  automatic  rigid  body  registration  was  used  to 


capture  the  global  transformation.  Features  included  a 
multi-resolution  approach,  two  similarity  measures,  and 
automatic  restarting  to  avoid  local  minimums.  Second,  local 
warping  registration  was  applied.  Interactively  placed  CP’s 
were  automatically  optimized  by  maximizing  the  MI  of 
corresponding  voxels  in  small  volumes  of  interest  and  by 
using  a  three  dimensional  thin  plate  spline  to  express  the 
deformation  throughout  the  image  volume.  More  than  100 
registration  experiments  with  17  MR  volume  pairs  deter¬ 
mined  the  quality  of  registration  under  conditions  simulat¬ 
ing  potential  interventional  MRI-guided  treatments  of 
prostate  cancer.  Evaluations  included  visual  inspection; 
voxel  gray  value  measures  such  as  MI,  CC,  and  intensity 
difference;  and  displacement  of  the  centroids  of  segmented 
prostates.  For  image  pairs  that  stress  rigid  body  registration 
(e.g.  supine,  the  diagnostic  position,  versus  legs  raised,  the 
treatment  position),  both  visual  and  numerical  evaluation 
methods  showed  that  warping  consistently  worked  better 
than  rigid  body.  Warping  registration  rectified  the  misalign¬ 
ment  in  the  pelvis  following  rigid  body  registration.  The 
prostate  centroid  displacement  for  a  typical  volume  pair  was 
reduced  from  3.4  to  0.6  mm  when  warping  was  added. 
Experiments  showed  that  ~  1 80  strategically  placed  CP’s 
were  sufficiently  expressive  to  capture  important  features  of 
the  deformation.  When  only  120  CP’s  were  used,  warping 
throughout  the  pelvis  was  visually  less  satisfactory  but  the 
prostate  was  aligned  reasonably  well.  For  volume  pairs  with 
images  acquired  in  the  same  position  (diagnosis -diagnosis) 
and  comparable  conditions,  the  rigid  body  method  worked 
sufficiently  well,  and  the  prostate  centroid  displacements 
were  <1.0  mm.  In  conclusion,  the  warping  registration 
method  works  better  than  rigid  body  registration  whenever 
patient  position  or  condition  is  greatly  changed  between 
acquisitions.  It  is  very  computational  efficient  for  hundreds 
of  CP’s  and  can  very  well  approximate  the  deformation  of 
the  pelvis  and  internal  organs.  It  will  probably  be  a  useful 
tool  for  many  applications. 
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ABSTRACT 

The  goal  of  this  research  is  to  register  real-time  interventional  magnetic  resonance  imaging  (iMRI) 
slice  images  with  a  previously  obtained  high-resolution  MRI  image  volume,  which  in  turn  can  be 
registered  with  functional  images  such  as  those  from  SPECT.  The  immediate  application  is  in 
iMRI-guided  treatment  of  prostate  cancer,  where  additional  images  are  desired  to  improve  tumor 
targeting.  In  this  article,  simulation  experiments  are  performed  to  demonstrate  the  feasibility  of 
slice-to-volume  registration  for  this  application.  We  acquired  3D  volume  images  from  a  1.5-T  MRI 
system  and  simulated  low-field  iMRI  image  slices  by  creating  thick  slices  and  adding  noise.  We 
created  a  slice-to-volume  mutual  information  registration  algorithm  with  special  features  to  improve 
robustness.  Features  included  a  multiresolution  approach,  two  similarity  measures,  and  automatic 
restarting  to  avoid  local  minima.  To  assess  the  quality  of  registration,  we  calculated  3D  displace¬ 
ments  on  a  voxel-by-voxel  basis  over  a  volume  of  interest  between  slice-to-volume  registration  and 
volume-to-volume  registration,  which  was  previously  shown  to  be  quite  accurate.  More  than  800 
registration  experiments  were  performed  on  MR  images  of  three  volunteers.  The  slice-to-volume 
registration  algorithm  was  very  robust  and  accurate  for  transverse  slice  images  covering  the  prostate, 
with  a  registration  error  of  only  0.4  ±  0.2  mm.  Error  was  greater  at  other  slice  orientations  and 
positions.  The  automatic  slice-to-volume  mutual  information  registration  algorithm  is  robust  and 
probably  sufficiently  accurate  to  aid  in  iMRI-guided  treatment  of  prostate  cancer.  Comp  Aid  Surg  7: 
257-267  (2002).  ©2003  Wiley-Liss,  Inc. 


Key  words:  image  registration;  mutual  information;  magnetic  resonance  imaging  (MRI);  interven¬ 
tional  MRI;  prostate  cancer;  minimally  invasive  treatment 


INTRODUCTION 

We  use  a  low-field  open  magnet  system  to  guide 
minimally  invasive  treatments,  including  radiofre¬ 
quency  (RF)  thermal  ablation  of  abdominal  can¬ 
cer.1"3  We  are  currently  investigating  the  incorpo¬ 


ration  of  other  medical  images  for  use  in  live-time 
treatment  planning  and  execution.  Examples  of  im¬ 
ages  for  possible  incorporation  include  high-reso¬ 
lution  MR  images  from  another  scanner  or  from  a 
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very-long-duration  acquisition  in  the  iMRI  scanner, 
MR  spectroscopy  images,  functional  MR  images, 
MR  angiography  images,  PET  images,  and  SPECT 
images.  In  some  cases,  these  other  images  can  be 
used  to  localize  a  disease  process  such  as  a  tumor; 
in  other  cases,  the  images  can  be  used  to  locate  a 
structure  that  should  be  avoided,  such  as  a  critical 
brain  structure  identified  with  functional  MRI. 
iMRI  is  currently  under  investigation  for  possible 
use  in  the  treatment  of  prostate  cancer.  Because 
MRI  does  not  reliably  show  prostate  tumor,  we 
intend  to  incorporate  nuclear  medicine  or  MR  spec- 
troscopy  images  with  an  improved  ability  to  detect 
and  localize  the  tumor.4’5 

To  incorporate  image  data  from  other  sources 
in  a  real-time  iMRI  procedure,  we  intend  to  register 
two-dimensional  (2D)  slice  images  acquired 
quickly  on  the  iMRI  scanner  with  a  previously 
acquired  volume  of  image  data.  An  image  volume 
from  another  modality  can  then  be  registered  with 
the  full  MR  volume.  Thus,  to  incorporate  SPECT  in 
an  iMRI  procedure,  we  will  first  register  the 
SPECT  image  volume  with  an  MR  volume.6  Then, 
when  registering  iMRI  slice  images  to  the  MR 
volume,  they  can  also  be  mapped  to  the  SPECT 
functional  image  data.  If  this  procedure  is  success¬ 
ful,  a  variety  of  potential  visualization  tools  will  be 
available  to  help  the  physician  localize  and  apply 
treatments  appropriately.  The  real-time  iMRI  im¬ 
ages  will  be  used  for  guidance,  and  any  small 
misregistration  errors  can  very  probably  be  men¬ 
tally  corrected  by  the  physician.  To  simplify  and 
improve  the  slice-to-volume  (SV)  registration  step, 
we  intend  to  always  use  MR  images  acquired  with 
similar  pulse  sequences.  In  this  report,  we  investi¬ 
gate  SV  registration  of  MR  images. 

Previous  success  with  registering  one  MR 
prostate  volume  to  another7  encourages  us  to  pur¬ 
sue  this  plan.  We  call  this  process  volume-to-vol- 
ume  or  VV  registration.  A  rigid-body  mutual  infor¬ 
mation  registration  method  was  used,  with  some 
features  to  improve  robustness.7  We  carefully  eval¬ 
uated  registration  quality  using  a  variety  of  meth¬ 
ods.  For  volume  pairs  acquired  over  a  short  time- 
span  from  a  supine  subject  with  legs  flat  on  the 
table,  registration  accuracy  for  both  prostate  cen¬ 
troids  (typically  <  1  mm)  and  bony  landmarks  (av¬ 
erage  1.6  mm)  was  in  the  order  of  one  voxel  (^1.4 
mm).  For  volumes  acquired  under  very  different 
conditions,  for  example,  with  legs  flat  or  raised  into 
the  treatment  position,  or  with  and  without  bladder 
or  rectal  filling,  we  obtained  somewhat  larger  pros¬ 
tate  centroid  registration  errors  of  about  3.0  mm. 
From  our  results  with  VV  prostate  registration,  it 


was  decided  that  SV  accuracy  could  be  assessed  by 
comparing  results  to  VV  registration  for  those  vol¬ 
ume  pairs  having  low  VV  registration  error. 

SV  registration  is  an  alternative,  fast  ap¬ 
proach  for  including  other  modalities  in  iMRI- 
guided  treatment.  Previously,  image  registration 
was  used  to  combine  preoperative  MRI  and  CT 
volumes  for  iMRI-guided  intranasal  microendos¬ 
copy  where  iMRI  volumes  were  acquired  and  reg¬ 
istered  with  preoperative  data.8  Here,  it  is  antici¬ 
pated  that  the  SV  method  will  be  faster,  because 
image  acquisition  is  faster  and  because  registration 
of  a  slice  should  require  many  fewer  calculations 
for  a  volume.  In  addition,  our  experience  is  that  2D 
slice  acquisitions  are  routinely  acquired  during 
iMRI  interventions,  whereas  volume  acquisitions 
are  only  acquired  infrequently.  In  previously  pub¬ 
lished  reports,  SV  registration  was  mainly  applied 
to  the  brain  for  applications  of  functional  MRI,9’10 
postmortem  pathology  studies,11  and  anatomical 
modeling.12  There  are  no  reports  of  SV  registration 
for  abdominal  organs  or  iMRI  guidance. 

The  application  of  SV  registration  methods  to 
iMRI-guided  treatment  of  prostate  cancer  raises 
several  challenges.  First,  iMRI  images  often  have  a 
lower  signal-to-noise  ratio  (SNR)  than  diagnostic 
MR  images  because  of  the  emphasis  on  fast  imag¬ 
ing  and  because  of  the  typically  lower  field- 
strength  of  open  iMRI  magnets.  Second,  a  single 
slice,  or  a  few  slices,  provides  many  fewer  struc¬ 
tures  than  an  entire  volume  for  voxel-based  match¬ 
ing.  Third,  the  prostate  can  move  relative  to  the 
pelvic  bones  due  to  changes  in  rectal  and  bladder 
filling13’14  or  changes  in  patient  posture  for  treat¬ 
ment 7  That  is,  alignment  of  the  pelvic  bones 
(prominent  anatomical  features  in  MR  gray-scale 
images)  does  not  necessarily  ensure  that  the  pros¬ 
tate  is  aligned.  Fourth,  the  normal  prostate  is  a 
small  organ;  when  healthy,  it  measures  only  —3.8 
cm  in  its  widest  dimension.15  The  prostate  is  lo¬ 
cated  below  the  much  larger  bladder,  which  can 
change  its  shape  and  size  during  imaging.  Finally, 
times  for  registration  and  algorithm  robustness  are 
of  particular  concern  in  this  application. 

In  this  study,  we  test  the  performance  of  SV 
registration.  In  the  next  section,  we  first  describe  a 
voxel-based  registration  method  with  special  fea¬ 
tures  to  improve  robustness.  This  is  followed  by 
details  of  how  entire  MR  volume  pairs  are  acquired 
on  a  conventional  MR  scanner  and  how  realistic 
iMRI  images  are  simulated.  Later,  results  are  pre¬ 
sented  from  over  800  registration  experiments 
comparing  slice-to-volume  with  volume-to-volume 
registration. 
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MATERIALS  AND  METHODS 
Registration  Algorithm 

We  used  an  algorithm  with  special  features  for  the 
slice-to-volume,  or  SV,  registration.  A  similar  algo¬ 
rithm  was  previously  successfully  applied  to  MR  vol- 
ume-to-volume  registration  of  the  prostate.7  The  al¬ 
gorithm  is  outlined  in  the  following  paragraphs.7*16 

We  used  two  similarity  measures,  mutual  in¬ 
formation  (MI)  and  correlation  coefficient  (CC),  in 
the  registration.  One  image,  R ,  is  the  reference ,  and 
the  other,  F ,  is  floating.  Their  mutual  information 
M1(R ,  F)  is  given  below.17 

The  joint  probability,  pRF(r ,  j),  and  the  mar¬ 
ginal  probabilities  of  the  reference  image,  pR(r ), 
and  the  floating  image,  pF(f),  can  be  estimated 
from  the  normalized  joint  intensity  histogram.  The 
correlation  coefficient  CC(R ,  F)  is  given  below.18 

CC(R ,  F) 

_  E(/?(r)  -  /?(r))(F(/)  -  F(/)) 

V2(*(r)  -  RirWWU)  -  F(/))2 

Here,  R(r),  F(f)  denote  the  average  intensi¬ 
ties  of  the  reference  and  floating  images,  respec¬ 
tively,  and  the  summation  includes  all  voxels 
within  the  overlap  region. 

We  compared  the  two  similarity  measures  at 
different  resolutions  to  determine  their  suitability 
for  SV  registration.  At  j  resolution,  we  resampled 
images  so  as  to  give  one-quarter  the  number  of 
voxels  along  each  linear  dimension.  At  full  resolu¬ 
tion ,  we  used  the  full  number  of  voxels.  In  Figure 
1,  the  two  similarity  measures  are  plotted  as  a 
function  of  translations.  Two  high-resolution  MR 
volumes  were  registered  as  described  previously,7 
and  values  were  plotted  with  the  origin  as  the 
optimal  transformation.  We  calculated  CC  and  MI 
values  while  moving  the  simulated  iMRI  image 
relative  to  the  high-resolution  MR  image  along  the 
coronal  (anterior-posterior)  axis.  The  simulated 
iMRI  image  was  obtained  as  described  later. 

We  use  a  multiresolution  approach  and  per¬ 
form  registration  from  low  to  high  resolution.  We 
use  CC  at  the  two  lower  resolutions  because  it 
gives  fewer  local  maxima  (Fig.  1)  and  can  be 
calculated  faster  than  MI.  We  use  MI  at  full  reso¬ 
lution  because  the  peaked  similarity  function  gives 


a  more  precise  solution  than  CC  (Fig.  1).  To  avoid 
local  maxima,  we  include  a  restarting  feature  where 
registration  is  restarted  with  randomly  perturbed 
parameters  obtained  from  a  uniform  distribution 
about  the  initial  transformation  values  at  the  current 
resolution  being  used.  The  algorithm  restarts  until 
the  absolute  CC  is  above  an  experimentally  deter¬ 
mined  threshold  or  the  maximum  number  of  re¬ 
starts  is  reached.  Absolute  CC  is  used  rather  than 
MI  because  it  has  a  well-defined  range  between  0 
and  1,  and  because  it  provides  an  independent 
check  of  the  MI  result  at  the  highest  resolution. 

All  important  results  are  recorded  following 
an  optimization  cycle,  including  the  CC  and/or  MI 
values,  the  number  of  restarts,  and  the  transforma¬ 
tion  parameters.  At  the  end  of  processing  at  a  lower 
resolution,  we  always  select  the  transformation  pa¬ 
rameters  having  the  maximum  CC  value.  We  then 
scale  the  translation  parameters  appropriately  and 
assign  the  new  parameters  to  be  initial  values  at  the 
next-higher  resolution.  At  the  highest  resolution, 
MI  rather  than  CC  is  the  similarity  measure,  and  we 
select  the  final  transformation  parameters  to  be 
those  with  the  maximum  MI  value. 

For  registration,  we  use  rigid-body  transfor¬ 
mation  (three  translations  and  three  rotations)  and 
trilinear  interpolation,  as  described  previously.19 
For  optimization,  we  use  the  downhill  simplex 
method  of  Nelder  and  Mead.20  Optimization  of 
similarity  ends  either  when  the  maximum  number 
of  calculations  is  reached  (typically  500)  or  when 
the  fractional  change  in  the  similarity  function  is 
smaller  than  a  tolerance  (typically  0.001).  There  are 
several  preprocessing  details  to  note:  The  input 
MRI  volume  is  a  3D  MR  acquisition  giving  256  X 
256  X  128  nearly  isotropic  voxels  over  a  field  of 
view  covering  the  whole  pelvis.  We  create  isotropic 
voxels  of  about  1.37  mm  on  each  side  using  3D 
linear  interpolation.  EDL  (Interactive  Data  Lan¬ 
guage,  Research  System  Inc.,  Boulder,  CO)  is  used 
as  the  programming  language. 

Typical  parameter  values  are  now  described. 
An  initial  guess  is  used  at  the  lowest  resolution  of 
all  zeros  because  the  patient  is  normally  oriented 
approximately  the  same  way  from  one  scan  to  the 
next.  All  CC  thresholds  are  set  at  0.5,  and  the 
maximum  number  of  restarts  is  set  at  20,  10,  and  5, 
from  low  to  high  resolution,  respectively. 

Image  Acquisition 

High-resolution  MRI  volumes  were  acquired  using 
a  1.5-T  Siemens  MRI  system  (Magnetom  Sym¬ 
phony,  Siemens  Medical  Systems,  Erlangen,  Germa¬ 
ny).  An  eight-element  phased-array  body  coil  was 
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Translation  (voxel)  Translation  (voxel) 

(c)  (d) 

Fig.  1.  Similarity  functions  are  plotted  as  a  function  of  translations  in  the  multiresolution  registration  process.  Two 
high-resolution  MRI  volumes  were  registered.  From  the  optimal  parameters,  we  computed  the  similarity  of  the  simulated  iMRI 
and  MRI  images  as  a  function  of  translations  along  the  coronal  (anterior-posterior)  axis.  MI  is  plotted  in  (a)  and  (c);  CC  is 
plotted  in  (b)  and  (d).  Graphs  (a)  and  (b)  are  at  the  lowest  resolution  where  images  are  down-sampled  by  1/4  along  each  linear 
dimension,  giving  a  distance  between  voxel  centers  of  —5.5  mm.  A  false  global  maximum  for  MI  occurred  at  +25  voxels. 
Graphs  (c)  and  (d)  are  plots  at  full  resolution.  Images  are  from  volunteer  S2. 


used  to  ensure  coverage  of  the  prostate  with  a  uniform 
sensitivity.  Typically,  two  anterior  and  two  posterior 
elements  were  enabled  for  signal  acquisition. 

Two  different  MR  sequences  were  used.  The 
first  was  a  3D  RF  spoiled  gradient  echo  steady-state 
pulse  sequence  (FLASH)  with  TR/TE/flip  parameters 
of  12/5.0/60,  giving  256  X  256  X  128  voxels  over  a 
330  X  330  X  256-mm  field  of  view  (FOV)  to  yield 
1.3  X  1.3  X  2.0-mm  voxels  oriented  to  give  the 
highest  resolution  for  transverse  slices.  The  acquisi¬ 
tion  time  is  5.6  min.  This  sequence  is  good  for  pelvic 
imaging,  but  is  not  ideal  for  the  prostate.  It  was  used 
to  acquire  volumes  for  volunteer  SI. 

The  second  sequence  was  a  3D  rapid  gradient 
echo  sequence  (PSIF)  designed  to  acquire  the  spin- 
echo  component  of  the  steady-state  response,  rather 
than  the  free  induction  decay.  The  spin-echo  com¬ 


ponent  forms  immediately  prior  to  the  RF  pulse, 
and  is  shifted  toward  the  prior  RF  pulse  through 
appropriate  gradient  waveform  design.  The  se¬ 
quence  with  9.4/5.0/60  (TR/TE/flip)  yields  160  X 
256  X  128  voxels  over  a  219  X  350  X  192-mm 
rectangular  FOV  and  1.4  X  1.4  X  1.5-mm  voxels 
oriented  to  give  the  highest  resolution  for  trans¬ 
verse  slices.  There  is  oversampling  at  31%  in  the 
slice  direction  to  reduce  aliasing  artifacts.  The  ac¬ 
quisition  time  is  4.3  min.  This  sequence  gave  ex¬ 
cellent  image  contrast  for  the  prostate,  and  was 
used  to  acquire  volumes  for  volunteers  S2  and  S3. 

Imaging  Experiments 

We  acquired  high-resolution  MRI  volumes  from 
three  volunteers.  For  each  volunteer,  three  image 
volumes  were  obtained  with  an  imaging  session. 
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Fig.  2.  Simulated  iMRI  images.  Images  on  the  left,  (a),  (c),  and  (e),  are  the  original  high-resolution  MR  images  in  the 
transverse,  coronal,  and  sagittal  planes,  respectively.  Images  on  the  right  are  corresponding  simulated  thick  iMRI  images  with 
SNR  =  10.  Images  are  from  volunteer  S2. 


Each  volume  was  acquired  with  compatible  condi¬ 
tions.  Volunteers  laid  supine  with  legs  flat,  similar 
to  the  position  in  routine  MR  scanning.  Between 
volume  acquisitions,  volunteers  got  off  the  MRI 
table,  stretched,  and  walked  around  to  ensure  that 
they  would  assume  a  different  position  when  they 
laid  back  on  the  table.  The  coil  array  was  centered 
on  the  prostate.  All  images  of  a  volunteer  were 
acquired  with  the  same  MRI  acquisition  parame¬ 
ters.  In  total,  there  were  nine  pairs  of  high-resolu- 
tion  MRI  volumes  for  registration. 

Simulation  of  iMRI  Slice  Images 

We  used  high-resolution  MRI  volumes  to  simulate 
iMRI  images  by  creating  thick  slices  and  adding 
noise  and  receive-coil  inhomogeneity.  Clinically, 
we  typically  use  an  iMRI  slice  thickness  of  4. 0-6.0 
mm.  We  averaged  three  slices  1.4  mm  thick  to 
create  a  4.2-mm-thick  slice. 

Noise  was  added  to  the  simulated  iMRI  im¬ 
age.  MR  noise  is  described  by  the  Rician  distribu¬ 
tion,21  but  at  reasonably  high  signal  values  it  is 
accurately  approximated  with  Gaussian  white 
noise.22  We  measured  typical  signal  and  noise  val¬ 
ues  on  our  open  magnet  system  using  a  homoge¬ 
nous  phantom  and  methods  described  else¬ 
where.23’24  Gaussian  noise  was  then  added  to  the 
simulated  iMRI  slice  images  either  to  match  the 
measured  SNR  or  to  give  much  greater  noise  to 
further  stress  registration.  We  report  noise  experi¬ 


ments  using  the  SNR  of  the  simulated  slice  images. 
Figure  2  shows  high-resolution  MRI  and  noisy 
simulated  iMRI  slice  images. 

We  simulated  receive-coil  inhomogeneity 
from  a  belt  coil  used  in  the  clinical  iMRI  acquisi¬ 
tions.  The  coil  is  modeled  as  a  solenoid,  and  the 
magnetic  field  is  highest  at  the  coil  center,  falling 
off  in  the  axial  direction.  According  to  the  Biot- 
Savart  law,25  this  model  also  accounts  for  the  spa¬ 
tial  sensitivity  of  the  coil  to  MR  signal  sources. 

Registration  Experiments 

We  desire  an  iMRI  slice  imaging  method  that  gives 
robust,  accurate  registrations,  and  is  relatively  in¬ 
sensitive  to  acquisition  parameters.  Experiments 
were  perfomed  to  determine  the  dependence  on 
slice  orientation  (transverse,  sagittal,  and  coronal), 
on  slice  position  relative  to  the  prostate  (above, 
centered,  and  below),  and  on  image  noise  from  fast 
imaging  techniques.  For  each  volume  pair,  we  ex¬ 
tracted  data  from  one  volume  and  registered  slice 
images  to  the  other  volume.  Many  different  slices 
were  used  in  experiments.  Slice  images  are  simu¬ 
lated  as  described  above. 

Registration  Evaluation 
Visual  Inspection 

Registration  experiments  were  evaluated  by  visual 
inspection.  We  used  RegViz,  a  program  created  in 
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EDL  in  our  laboratory  with  multiple  visualization 
and  analysis  methods.  First,  we  manually  seg¬ 
mented  prostate  boundaries  in  image  slices  and 
copied  them  to  corresponding  slices.  This  enabled 
visual  determination  of  the  overlap  of  prostate 
boundaries  over  the  entire  volume.  Second,  color 
overlay  displays  were  used  to  evaluate  overlap  of 
structures.  One  image  was  rendered  in  gray  and  the 
other  in  the  “hot  iron”  color  scheme  available  in 
IDL.  To  visualize  potential  differences,  it  was  use¬ 
ful  to  interactively  change  the  contribution  of  each 
image  using  the  transparency  scale.  Third,  we  used 
a  sector  display  that  divided  the  reference  and  reg¬ 
istered  images  into  rectangular  sectors  and  created 
an  output  image  by  alternating  sectors  from  the  two 
input  images.  This  way,  even  subtle  shifts  of  edges 
would  be  clearly  seen. 

Comparison  to  Volume-to-Volume 
Registration  Standard 

Our  standard  evaluation  method  was  to  compare 
SV  and  VV  registration.  Because  this  relies  on  VV 
registration  accuracy,  we  will  now  review  our  pre¬ 
vious  results.7  For  volume  pairs  acquired  over  a 
short  time-span  from  a  supine  subject  with  legs  flat 
on  the  table,  prostates  were  well  aligned,  and  pros¬ 
tate  centroid  displacements  were  typically  <1  mm. 
The  registration  accuracy,  as  determined  from  dis¬ 
placements  of  pelvic  bony  landmarks,  was  1.6  ± 
0.2  mm.  This  error  might  be  overestimated  because 
it  includes  the  uncertainty  of  locating  the  bony 
landmarks.  The  centroid  error  was  slightly  smaller 
because  the  prostate  was  at  the  volume  center  and 
rotation  errors  had  less  effect.  From  our  success 
with  VV  prostate  registration,  it  was  decided  that 
we  could  obtain  SV  accuracy  by  comparison  with 
VV  registrations  for  those  volume  pairs  having  low 
VV  registration  error. 

To  compare  SV  and  VV  registration,  we 
defined  a  rectangular  volume  of  interest  (VOI) 
that  just  covered  the  prostate  and  calculated 
voxel  displacements  between  the  two  registra¬ 
tions.  To  voxels  within  this  VOI,  we  applied  the 
transformations  obtained  by  VV  and  SV  regis¬ 
trations.  We  then  calculated  the  3D  displace¬ 
ments  between  the  transformed  voxels.  The  mean 
voxel  distance  was  used  as  the  metric  of  SV 
registration  error.  For  the  evaluation  of  algorithm 
robustness,  we  defined  the  SV  registration  as 
being  successful  when  the  3D  displacement  was 
less  than  2.0  mm. 


Fig.  3.  Sector  display  showing  excellent  pelvic  registra¬ 
tion.  Image  (a)  is  a  transverse  slice  from  a  high-resolution 
MRI  volume.  Image  (b)  is  the  corresponding  noise  iMRI 
slice.  In  the  sector  display  shown  in  (c),  alternating  rectan¬ 
gular  sections  from  (b)  are  made  brighter  and  combined 
with  sections  from  (a)  to  show  the  matching  boundaries. 
The  boundaries  of  bones  and  other  structures  are  continu¬ 
ous,  as  shown  particularly  at  locations  1-4.  Other  slices 
from  this  volume  were  also  perfectly  aligned,  indicating 
good  3D  alignment.  Image  volumes  are  from  Volunteer  S2. 

RESULTS 

Slice  Orientation 

In  Figure  3,  the  sector  display  shows  a  simulated 
slice  image  registered  with  a  high-resolution  vol¬ 
ume  image.  The  slice  image  was  obtained  at  a 
transverse  orientation  near  the  center  of  the  pros¬ 
tate.  The  sector  display  shows  excellent  alignment 
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Fig.  4.  SV  registration  using  slices  at  different  orientations.  The  error  metric  is  the  voxel  displacement  between  the  SV  and 
VV  transformations.  Plotted  are  mean  errors,  as  well  as  maxima  and  minima  over  a  rectangular  VOI  surrounding  the  prostate. 
For  each  volunteer,  SI,  S2,  and  S3,  we  registered  three  volume  pairs.  For  each  pair,  five  registration  experiments  were 
conducted  using  five  different  simulated  iMRI  transverse  slices  intersecting  the  prostate.  The  simulated  SNR  was  25. 
Averaging  data  across  all  volunteers  gives  0.4  ±  0.2  mm,  0.5  ±  0.2  mm,  and  2.6  ±  1.6  mm  errors  for  transverse,  coronal,  and 
sagittal  slices,  respectively. 


Transverse  slice  Coronal  slice  Sagittal  slice 


at  this  position.  Other  transverse  images  were  also 
well  aligned,  indicating  that  the  registration  was 
successful  in  three  dimensions. 

Figure  4  shows  results  for  single  slices  oriented 
in  the  three  traditional  orthogonal  directions.  All 
slices  are  near  the  center  of  the  prostate.  Compared  to 
W  transformations,  registration  error  is  smallest  for 
transverse  slices  and  largest  for  sagittal  slices.  Aver¬ 
aging  across  all  transverse  data,  the  error  is  only  0.4  ± 
0.2  mm,  where  the  latter  number  is  a  standard  devi¬ 
ation.  Coronal  slices  also  gave  quite  small  errors  of 
0.5  ±  0.2  mm.  As  shown  in  Figure  5c,  transverse 
slices  work  best  because  they  contain  abundant  ana¬ 
tomical  structures  that  do  not  deform  relative  to  the 
prostate.  That  is,  a  transverse  slice  centered  at  the 
prostate  excludes  the  bladder  that  can  deform  and 
create  an  inconsistent  match  for  registration.  Figure 
5a,b  shows  that  coronal  and  sagittal  views  contain 
large  regions  of  the  bladder  and  rectum  that  can 
deform  with  filling.  The  following  analyses  are  all 
based  on  transverse  slices. 

Slice  Position 

Figure  6  shows  registration  results  for  transverse 
slices  at  different  distances  from  the  prostate  cen¬ 
ter.  Slices  centered  on  the  prostate  produced  the 
best  results,  with  a  displacement  error  always  less 


than  1.0  mm.  The  reason  is  that  slices  centered  at 
the  prostate  include  an  abundance  of  bony  struc¬ 
tures  giving  good  information  for  registration.  They 
also  exclude  portions  of  the  bladder  that  can  de¬ 
form  and  create  inconsistent  matches  for  registra¬ 
tion  (Fig.  7).  Slices  above  the  prostate  include  the 
deformable  bladder,  which  can  stress  the  registra¬ 
tion  algorithm,  especially  for  volunteer  S3  (Fig.  6). 
Slices  below  the  prostate  mainly  contain  muscle 
and  fatty  regions  from  the  hips  that  can  deform,  and 
there  is  less  information  for  rigid-body  registration. 
This  effect  was  more  pronounced  for  volunteer  S2 
(Fig.  6). 

Noise  Level 

Figure  8  shows  registration  results  for  transverse 
slices  with  added  noise.  The  typical  iMRI  SNR 
under  clinical  conditions  is  about  25.  Even  when 
noise  far  exceeded  this  normal  situation,  regis¬ 
tration  results  were  still  quite  good.  A  100% 
success  rate  was  achieved  with  an  acceptance 
criterion  of  <2.0  mm,  even  when  the  SNR  was  as 
bad  as  10. 

Robustness  and  Calculation  Time 

The  registration  algorithm  was  quite  robust  for 
transverse  slices  covering  the  prostate.  Using  the 
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Fig.  5.  Image  slices  showing  the  advantage  of  transverse 
images.  The  coronal  slice  in  (a)  contains  the  bladder,  as 
indicated  by  the  black  arrow.  The  sagittal  slice  in  (b)  con¬ 
tains  both  the  bladder  (vertical  arrow)  and  rectum  (horizon¬ 
tal  arrow).  The  transverse  slice  in  (c)  excludes  the  bladder 
and  contains  abundant  bony  structures. 


nine  volume  pairs  from  three  volunteers,  the 
algorithm  never  failed  for  any  transverse  slice 
covering  the  prostate.  In  addition,  the  final  reg¬ 
istration  result  was  insensitive  to  initial  guesses 
within  a  large  range:  [—60,  +60]  mm  for  trans¬ 
lations  and  [-20,  +20]  degrees  for  rotations. 
With  the  restarting  algorithm,  we  even  success¬ 
fully  registered  slices  as  much  as  80  mm  from  the 
optimum.  This  working  range  is  probably  suffi¬ 
cient  for  clinical  applications  where  we  can  en¬ 
sure  good  starting  values.  Using  the  pelvic  bones 
as  markers  and  needle  localization  methods,26  we 
should  be  able  to  position  the  prostate  within 
about  ±20  mm.  In  addition,  the  patient  normally 
lies  supine  in  the  MR  bed  with  very  little  rotation 
(<±5  degrees). 

The  time  for  an  SV  registration  was  typically 
about  5  s  on  a  Pentium  IV  1.8-GHz  CPU  with  1  GB 
of  memory.  The  algorithm  was  written  in  IDL 
and  could  probably  be  made  much  faster  in  a 
lower-level  language  such  as  C.  A  call  to  the  sim¬ 
plex  optimization  typically  resulted  in  50  to  105 
similarity  evaluations  before  the  tolerance  value 
(0.001)  was  reached. 

DISCUSSION 

For  transverse  slices  covering  the  prostate,  the 
slice-to-volume  registration  results  agreed  very  fa¬ 
vorably  with  the  volume-to-volume  results.  That  is, 


Fig.  6.  SV  registration  using  transverse  slices  at  different  positions.  Groups  of  five  iMRI  slices  each  were  extracted  near  the 
prostate  center,  —35  mm  above  the  prostate  base,  and  —35  mm  below  the  prostate  apex  and  registered  to  the  MR  volume. 
Other  details  are  given  in  Figure  4.  Slices  at  the  prostate  center  worked  best. 
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Fig.  7.  Transverse  images  at  different  positions  relative  to 
the  prostate.  The  image  35  mm  above  the  prostate  in  (a) 
includes  the  bladder,  indicated  by  the  white  arrow.  A  slice 
35  mm  below  the  prostate  in  (b)  mainly  contains  muscle 
(dark)  and  fatty  regions  (white).  A  slice  centered  at  the 
prostate  in  (c)  has  abundant  bony  structures. 


we  found  that  SV  is  equal  to  VV  registration,  with 
an  average  voxel  displacement  between  them  of 
only  0.4  mm  in  the  prostate.  The  accuracy  of  SV  is 
essentially  the  same  as  that  previously  reported  for 
VV  registration.7  In  our  previous  report,  it  was 
found  that,  whenever  the  subject  was  in  comparable 
conditions,  the  prostate  centroid  registration  error 
was  typically  <1  mm,  or  less  than  a  voxel  of  1.4 
mm.  Hence,  for  SV  applied  to  images  in  this  report, 
we  predict  a  prostate  error  of  very  nearly  1.4  mm  or 
less.  In  the  previous  report,  it  was  found  that  when¬ 


ever  supine  subjects  raised  their  legs,  the  prostate 
moved  towards  the  posterior  direction  by  about  3 
mm.7  Another  factor  that  affects  prostate  registra¬ 
tion  is  rectal  filling.7’13  Hence,  we  recommend  that 
prostate  registration  be  done  with  the  patient  under 
similar  conditions  by  maintaining  a  similar  posture 
and  by  taking  clinical  measures  to  reduce  rectal  and 
bladder  filling.  We  see  no  reason  to  suspect  that  SV 
registration  would  not  be  very  accurate  in  these 
cases. 

Slice-to-volume  registration  is  probably  suf¬ 
ficiently  accurate  for  many  iMRI  applications. 
Compared  to  a  typical  iMRI  slice  thickness  of  ~3.0 
mm,  SV  registration  is  quite  accurate.  The  accuracy 
of  SV  is  probably  much  better  than  that  of  multi¬ 
modality  registration,  where  the  typical  functional 
image  has  a  thickness  of  3.0-4.0  mm.  If  one  were 
to  use  functional  or  high-resolution  MR  images 
directly  for  targeting,  the  requirements  for  registra¬ 
tion  accuracy  would  be  great.  However,  fused  im¬ 
age  data  will  not  be  used  blindly.  Rather,  one  can 
use  fused  images  as  a  guide.  Physicians  will  always 
use  the  real-time  anatomical  iMRI  images  for  nee¬ 
dle  guidance.26’27  With  proper  visualization  tools, 
they  should  be  able  to  mentally  account  for  any 
small  registration  errors.  In  addition,  there  is  very 
often  image  evidence  of  cancer  in  MR  prostate 
images  that  can  perhaps  be  identified  with  the  aid 
of  functional  images.  Such  MR-visible  lesions  can 
become  the  markers  for  tumor  targeting.  As  a  result 
of  these  considerations,  we  believe  that  the  accu¬ 
racy  of  SV  registration  is  sufficient  to  justify  fur¬ 
ther  investigation  of  its  application  in  iMRI. 

The  algorithm  is  quite  robust  for  the  SV  reg¬ 
istration.  Significantly,  the  registration  never  failed 
for  transverse  slices  covering  the  prostate.  It  was 
obtained  even  in  the  presence  of  noise  levels  far 
beyond  those  encountered  in  iMRI.  This  is  very 
important  for  iMRI,  where  magnetic  fields  are  low 
and  imaging  is  fast.  It  was  also  determined  that 
coronal  images  work  fairly  well.  This  could  be 
useful,  because  two  orientations  might  provide 
more  flexibility  for  clinical  applications. 

There  are  several  reasons  for  the  robustness. 
First,  using  both  CC  and  MI  at  different  resolutions 
was  an  important  feature  that  increased  robustness. 
When  only  MI  was  used,  registrations  at  low  res¬ 
olution  very  often  gave  false  solutions  that  misled 
registration  at  the  next-highest  resolution.  Fortu¬ 
nately,  CC  performed  well  and  gave  many  fewer 
local  maxima  at  low  resolution.7  However,  MI  gave 
a  more  accurate  solution  at  high  resolution  due  to 
the  peaked  MI  surface.7  Our  registration  algorithm 
combined  advantages  from  the  two  similarity  mea- 
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Fig.  8.  SV  registration  at  different  noise  levels.  The  simulated  iMRI  image  slice  had  SNR  values  of  20,  10,  5,  and  3, 
corresponding  to  added  noise  levels  of  4,  8,  16,  and  27  standard  deviation  gray  levels,  respectively.  The  transverse  slices  from 
Figure  4  were  used.  Other  details  are  given  in  the  legend  of  Figure  4.  Even  though  noise  greatly  exceeds  that  found  in  iMRI, 
registration  was  little  affected. 


sures.  Second,  the  restarting  mechanism  was  also 
quite  important.  Without  restarting,  it  was  found 
that  registrations  sometimes  failed  in  cases  of  vol¬ 
umes  with  a  large  mismatch  of  54  mm  and  high 
noise.  Even  these  cases  resulted  in  a  proper  solution 
when  restarting  was  employed. 

The  simulation  provided  a  realistic,  simple, 
and  efficient  way  to  evaluate  the  algorithm  for  our 
application.  The  simulated  iMRI  images  are  good 
representations  of  images  from  our  low-field- 
strength  iMRI  system.  However,  there  are  other 
practical  aspects  of  iMRI  imaging  that  were  not 
covered  in  the  simulations.  For  instance,  we  did  not 
reproduce  the  susceptibility  artifact  of  an  RF  treat¬ 
ment  needle  in  our  simulations.  From  previous 
experience,19  we  think  this  is  not  a  problem,  be¬ 
cause  the  needle  artifact  occupies  a  relatively  small 
percentage  of  voxels  in  most  cases. 

Another  practical  aspect  is  possible  deforma¬ 
tion  in  the  pelvic  region.  When  images  were  ac¬ 
quired  under  much  different  conditions,  such  as 
supine  or  with  legs  raised,  it  was  determined  that 
warping  was  required  to  successfully  register  the 
prostate.28-29  The  warping  registration  method  was 
based  upon  independent  optimization  of  many  in¬ 
teractively  placed  control  points  using  MI  and  a 
thin-plate  spline  transformation.  About  180  strate¬ 
gically  placed  control  points  were  sufficiently  ex¬ 


pressive  to  capture  important  features  of  the  defor¬ 
mation.  In  the  event  that  a  device  such  as  an 
endorectal  MR  coil  is  used,  warping  registration 
will  be  required. 

Finally,  we  conclude  that  the  slice-to-volume 
registration  algorithm  is  quite  robust  for  transverse 
slice  images  covering  the  prostate,  and  that  regis¬ 
tration  accuracy  is  probably  sufficiently  accurate  to 
aid  iMRI-guided  thermal  ablation  of  prostate  can¬ 
cer.  It  is  quite  feasible  to  include  previously  ac¬ 
quired  high-resolution  MRI  or  nuclear  images  in 
iMRI-guided  treatment  procedures,  and  we  are  be¬ 
ginning  to  explore  this  application  in  animal  exper¬ 
iments. 
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Abstract 

A  three-dimensional  (3D)  mutual  information  registration  method  was  created 
and  used  to  register  MRI  volumes  of  the  pelvis  and  prostate.  It  had  special 
features  to  improve  robustness.  First,  it  used  a  multi-resolution  approach 
and  performed  registration  from  low  to  high  resolution.  Second,  it  used  two 
similarity  measures,  correlation  coefficient  at  lower  resolutions  and  mutual 
information  at  full  resolution,  because  of  their  particular  advantages.  Third, 
we  created  a  method  to  avoid  local  minima  by  restarting  the  registration 
with  randomly  perturbed  parameters.  The  criterion  for  restarting  was  a 
correlation  coefficient  below  an  empirically  determined  threshold.  Experiments 
determined  the  accuracy  of  registration  under  conditions  found  in  potential 
applications  in  prostate  cancer  diagnosis,  staging,  treatment  and  interventional 
MRI  (iMRI)  guided  therapies.  Images  were  acquired  in  the  diagnostic  (supine) 
and  treatment  position  (supine  with  legs  raised).  Images  were  also  acquired  as 
a  function  of  bladder  filling  and  the  time  interval  between  imaging  sessions. 
Overall  studies  on  three  patients  and  three  healthy  volunteers,  when  both 
volumes  in  a  pair  were  obtained  in  the  diagnostic  position  under  comparable 
conditions,  bony  landmarks  and  prostate  3D  centroids  were  aligned  within 
1.6  ±  0.2  mm  and  1.4  zb  0.2  mm,  respectively,  values  only  slightly  larger 
than  a  voxel.  Analysis  suggests  that  actual  errors  are  smaller  because  of 
the  uncertainty  in  landmark  localization  and  prostate  segmentation.  Between 
the  diagnostic  and  treatment  positions,  bony  landmarks  continued  to  register 
well,  but  prostate  centroids  moved  towards  the  posterior  2. 8-3. 4  mm.  Manual 
cropping  to  remove  voxels  in  the  legs  was  necessary  to  register  these  images.  In 
conclusion,  automatic,  rigid  body  registration  is  probably  sufficiently  accurate 
for  many  applications  in  prostate  cancer.  For  potential  iMRI-guided  treatments, 
the  small  prostate  displacement  between  the  diagnostic  and  treatment  positions 
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can  probably  be  avoided  by  acquiring  volumes  in  similar  positions  and  by 
reducing  bladder  and  rectal  volumes. 


1.  Introduction 

We  are  investigating  three-dimensional  (3D)  image  registration  to  be  used  in  applications 
of  prostate  cancer  diagnosis,  staging  and  therapy.  In  particular,  we  are  interested  in 
applications  related  to  the  minimally  invasive  interventional  MRI  (iMRI)  guided  treatment 
of  prostate  cancer.  Our  group  currently  uses  iMRI  on  a  low-field  open  magnet  system  to 
guide  radiofrequency  (RF)  thermal  ablation  of  abdominal  cancer  (Lewin  et  al  1998),  and  we 
are  investigating  this  method  for  prostate  cancer  treatment.  A  unique  feature  of  iMRI-guided 
thermal  ablation  is  that  therapy  can  be  monitored  either  by  acquiring  images  of  the  thermally 
induced  lesion  or  by  measuring  temperature.  In  addition,  MR  imaging  of  the  prostate  is 
desirable  because  it  more  accurately  delineates  the  prostate  than  does  CT  (Milosevic  et  al 
1998),  which  can  overestimate  the  prostate  volume  (Roach  et  al  1996),  and  ultrasound,  which 
has  a  tendency  to  underestimate  the  extent  of  lesions  (Boni  et  al  1995). 

Several  important  applications  require  registration  of  images  of  the  prostate.  First, 
comparison  of  registered  MR  images  acquired  before  and  immediately  after  RF  ablation 
can  be  used  to  determine  whether  a  tumour  is  adequately  treated.  This  is  particularly 
helpful  in  instances  where  the  edematous  response  to  treatment  can  be  confused  with  a 
highly  perfused  tumour.  Second,  registration  of  serial  examinations  can  be  used  to  follow 
regression/progression  of  tumour.  Third,  registration  of  functional,  biochemical  images  such 
as  single  photon  emission  computed  tomography  (SPECT),  positron  emission  tomography 
(PET)  and  MR  spectroscopy  with  anatomical  MR  images  is  useful  for  detecting  and  localizing 
cancer.  Fourth,  incorporating  the  functional,  biochemical  images  into  the  iMRI  paradigm  will 
aid  image-guided  treatments.  Fifth,  on  a  low-field  magnetic  system  during  iMRI  treatments 
where  fast  imaging  is  important,  it  might  be  highly  desirable  to  register  high-quality  MR  image 
from  a  conventional  MR  scanner  to  the  live- time  iMRI  images  (Fei  et  al  2001).  In  this  study, 
we  investigate  registration  of  high-resolution  MR  volumes.  Multi-modality  image  registration 
results  were  reported  elsewhere  (Fei  et  al  2001,  Lee  et  al  2000,  2001a). 

Many  reports  describe  methods  and  evaluations  for  registration  in  the  brain  (Hill  et  al 
200 1 );  far  fewer  describe  results  for  the  pelvis  or  prostate.  For  example,  manual  registration  has 
been  used  where  an  operator  cues  on  segmented  vascular  structures  (Hamilton  et  al  1999)  or 
other  anatomical  landmarks  (Balter  et  al  1995,  Liehn  et  al  1992,  Narayana  et  al  1997).  Others 
have  used  automated  3D  schemes  that  match  contours  of  bones  and  sometimes  other  structures 
that  are  extracted  using  manual  or  interactive  segmentation  (Antolak  et  al  1998,  Herk  et  al 
1998,  Remeijer  et  al  2000).  Manual  segmentation  has  also  been  used  to  create  surfaces  for 
automatic  registration  (Roeskec/a/  1995,  Scott  et  al  1994).  All  of  these  methods  require 
either  segmentation  or  visual  identification  of  structures.  Voxel-based  methods,  particularly 
those  based  upon  mutual  information,  are  robust,  require  no  segmentation  that  can  be  prone 
to  error,  are  suitable  for  multi-modality  registration,  are  highly  accurate  for  brain  registration 
(Maes  et  al  1997),  and  are  suitable  for  abdominal  registration  (Carrillo  et  al  2000).  There 
are  no  reports  of  using  such  methods  for  pelvis  and/or  prostate  registration.  For  registration 
of  brain  and  other  organs,  registration  accuracy  has  been  assessed  using  fiducial  markers 
(Maurer  e/  al  1997,  Wan getal  1996)  and  anatomical  landmarks  (Fitzpatrick  et  al  1998, 
Peters  et  al  2000,  Wilson  et  al  1998). 
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There  are  challenges  for  registration  in  the  pelvis  and  prostate  that  might  reduce 
the  effectiveness  of  automatic  voxel-based  registration.  First,  the  abdomen  has  irregular 
boundaries,  unlike  the  head  to  which  registration  has  been  most  often  applied.  Second,  the 
normal  prostate  is  a  small  organ,  which  when  healthy,  measures  only  about  38.0  mm  in  its 
widest  dimension  transversely  across  the  base  (Gray  1977).  Third,  there  are  potential  factors 
such  as  different  patient  positions,  and  rectal  and  bladder  filling  (Herk  etal  1995)  that  can 
stress  registration.  In  addition,  it  is  more  difficult  to  evaluate  pelvic  and/or  prostate  registration 
because  no  external  markers  are  available. 

In  the  present  study,  we  perform  experiments  to  determine  the  potential  accuracy  of 
registering  prostate  MR  images  using  a  modified  mutual  information  algorithm  that  uses  rigid- 
body  transformations.  High-quality,  3D  MR  image  volumes  from  a  commercially  available 
1.5  T  system  are  used  to  determine  the  best  possible  results.  We  examine  conditions  found 
in  potential  applications  described  previously.  We  develop  and  use  a  variety  of  assessment 
methods  that  include  measuring  displacements  of  bony  landmarks  and  of  the  segmented 
prostate.  One  goal  is  to  obtain  baseline  accuracy  measurements  for  planning  future  applications 
of  registration  in  prostate  cancer  management. 


2.  Methods 

2.1.  Data  acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system  (Magnetom  Symphony, 
Siemens  Medical  Systems,  Erlangen,  Germany).  An  8-element  phased  array  body  coil  was 
used  to  ensure  coverage  of  the  prostate  with  a  uniform  sensitivity.  Typically,  two  anterior 
and  two  posterior  elements  were  enabled  for  signal  acquisition.  We  used  two  different 
MR  sequences.  First,  we  used  a  3D  RF  spoiled  gradient  echo  steady-state  pulse  sequence 
(FLASH)  with  TR/TE/flip  parameters  of  12/5.0/60  which  give  256  x  256  x  128  voxels  over  a 
330  x  330  x  256  mm  field  of  view  (FOV)  to  yield  1.29  x  1.29  x  2.0  mm  voxels  oriented 
to  give  the  highest  resolution  for  transverse  slices.  The  acquisition  time  was  5.63  min.  This 
sequence  was  good  for  pelvic  imaging  but  was  not  ideal  for  the  prostate.  Second,  we  used 
a  3D  rapid  gradient  echo  sequence  (PSIF)  designed  to  acquire  the  spin-echo  component  of 
the  steady-state  response,  rather  than  the  free  induction  decay.  The  spin-echo  component 
was  formed  immediately  prior  to  the  RF  pulse  and  it  was  shifted  towards  the  prior  RF  pulse 
through  appropriate  gradient  waveform  design.  The  sequence  with  9.4/5.0/60  (TR/TE/flip) 
yielded  160  x  256  xl28  voxels  over  a  219  x  350  x  192  mm  rectangular  FOV  and  1.37  x 
1 .37  x  1 .5  mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices.  There  was 
over  sampling  at  31%  in  the  slice  direction  to  reduce  aliasing  artifacts.  The  acquisition  time 
was  4.25  min.  Most  often,  we  used  the  second  sequence,  which  gave  excellent  image  contrast 
for  the  prostate  and  its  surroundings. 

2.2.  Image  volumes  for  registration 

We  acquired  3D  MRI  volume  images  from  three  prostate  cancer  patients  and  three  normal 
volunteers  under  four  conditions  simulating  anticipated  situations  in  diagnostic  and  treatment 
applications.  They  are  diagnostic  position ,  treatment  position ,  empty  bladder  and  diagnosis 
1  week.  In  the  diagnostic  position,  the  subject  lay  supine  throughout  MR  scanning.  The 
reference  volume  was  always  obtained  in  the  diagnostic  position.  In  the  treatment  position, 
the  subject  was  supine  and  his  legs  were  supported  at  30°-60°  relative  to  the  horizontal 
position  and  separated  in  a  ‘V’  with  an  angle  of  60°-90°  between  the  legs.  This  is  similar 


826 


B  Fei  et  al 


to  the  lithotomy  position  used  in  some  prostate  therapies  and  it  should  provide  access  for 
needle  insertion  in  brachytherapy  or  RF  thermal  ablation.  In  some  experiments,  the  subject 
micturated  to  create  an  empty  bladder  prior  to  imaging.  For  each  subject,  volumes  were 
typically  obtained  within  an  imaging  session  of  1-2  h.  We  imaged  one  subject  (V2)  a  week 
before  the  standard  imaging  session  and  we  refer  to  this  volume  as  diagnosis  1  week.  Between 
volume  acquisitions,  subjects  got  off  the  MRI  table,  stretched  and  walked  around  to  ensure 
that  they  would  assume  a  different  position  on  the  table.  The  coil  array  was  centred  on  the 
prostate.  All  images  of  a  subject  were  acquired  using  the  same  pulse  sequence  and  acquisition 
parameters  so  as  to  ensure  very  similar  grey  values.  In  total,  we  registered  22  volume  pairs 
consisting  of  one  pair  for  each  patient,  six  pairs  for  each  volunteer  and  one  additional  pair  for 
volunteer  V2. 

There  are  several  preprocessing  details.  Isotropic  voxels  are  created  using  3D  linear 
interpolation  or  higher  order  interpolation  methods  (Carrillo  et  al  2000).  From  the  top  and 
bottom  of  the  volume,  we  optionally  crop  transverse  slices  that  are  over  35  mm  away  from  the 
prostate  rim.  Cropping  is  done  to  remove  slices  having  reduced  brightness  due  to  sensitivity 
fall  off  from  the  receiver  coils,  artifacts  from  a  small  field  of  view,  displacement  of  the  legs  in 
the  treatment  position,  and/or  bladder  deformation. 


23.  Similarity  measurements 


We  used  two  similarity  measures,  mutual  information  (MI)  and  correlation  coefficient  (CC), 
in  our  registration.  Suppose  one  volume  R  is  the  reference ,  and  the  other  F  is  floating.  Their 
mutual  information  MI (R,F)  is  given  below  (Maes  et  al  1997): 


MI(/?,F)  =  ^/>*F(r,/)  log 

r.f 


PrfO\  f) 
PR(r)  ■  Pf(I) 


The  joint  probability  pFF(r,  /)  and  the  marginal  probabilities  /?F(r)  of  the  reference  image 
and  pf(/)  of  the  floating  image,  can  be  estimated  from  the  normalized  joint  and  marginal 
intensity  histogram,  respectively.  The  correlation  coefficient  CC(R,  F)  is  given  below  (Press 
et  al  1993): 

^,/n  Z(R(0  ~  R(r))(F(f)  -  F(f)) 

r  )  - - 

V£ (*(>*)  “  Hr))2  £(F(/)  -  F(/))2 

Here  R(r ),  F(  f)  denote  the  average  intensities  of  the  reference  and  floating  volumes  and  the 
summation  includes  all  voxels  within  the  overlap  of  both  volumes. 


2.4.  Registration  algorithm  with  special  features 

The  registration  algorithm  includes  special  features  to  improve  the  robustness  for  MR  pelvic 
images.  We  use  a  multi-resolution  approach  and  perform  registration  from  low  to  high 
resolution.  At  low  resolution,  we  resample  both  images  at  1  /4  or  1  /2  number  of  voxels  along 
each  linear  dimension,  respectively.  Iterative  optimization  of  the  similarity  is  used  to  vary 
the  six  rigid-body  transformation  parameters  (three  translations  and  three  angles).  We  use  the 
correlation  coefficient  at  the  two  lower  resolutions  because  it  gives  fewer  local  maximums  and 
because  it  can  be  calculated  faster  than  MI.  We  use  MI  at  full  resolution  because  the  peaked 
similarity  function  gives  a  more  precise  solution  than  CC. 

We  created  a  method  to  avoid  local  minima  by  restarting  the  registration  with  randomly 
perturbed  parameters  obtained  from  a  uniform  distribution  about  the  very  first  initial  guess  at 
each  resolution.  The  distribution  was  centred  on  the  initial  guess  because  we  wanted  to  use  the 
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best  solution  from  the  lower  resolution.  The  algorithm  restarts  until  the  absolute  correlation 
coefficient  between  the  reference  and  registered  volumes  is  above  a  threshold  or  the  maximum 
number  of  restarts  is  reached.  The  perturbation  range  is  ±5°  and  ±5  voxels  corresponding 
to  ±27.3,  ±13.7  or  ±6.8  mm  for  resolutions  1  /4,  1  /2,  or  full  voxels,  respectively.  Absolute 
CC  is  used  for  the  restart  test  rather  than  MI  because  CC  has  a  well-defined  range  between  0 
and  1,  because  CC  provides  an  independent  check  of  the  MI  result,  and  because,  as  described 
later,  CC  has  fewer  problems  with  local  and  incorrect  global  maximums  for  registrations  at 
low  resolution  far  from  the  optimum  value. 

We  record  all  important  results  following  an  optimization  cycle  including  the  CC  and/or 
MI  values,  the  number  of  restarts  and  the  transformation  parameters.  At  the  end  of  processing 
at  a  lower  resolution,  we  always  select  the  transformation  parameters  having  the  maximum  CC 
value.  We  then  scale  the  translation  parameters  appropriately  and  assign  the  new  parameters 
to  be  initial  values  at  the  next  higher  resolution.  At  the  highest  resolution,  we  select  the  final 
transformation  parameters  to  be  those  with  the  maximum  MI  value. 

There  are  several  implementation  details.  We  used  rigid-body  transformation  (three 
translations  and  three  angles)  and  trilinear  interpolation.  For  optimization,  we  use  the  downhill 
simplex  method  of  Nelder  and  Mead  (1965)  and  the  Powell  method  (1962),  but  we  prefer  the 
former  method  as  described  later.  Optimization  of  alignment  ends  either  when  the  maximum 
number  of  MI  calculations  is  reached  (typically  500)  or  the  fractional  change  in  MI  is  smaller 
than  a  tolerance  (typically  0.001).  For  the  22  volume  pairs  reported  here,  the  maximum 
number  of  calculations  was  reached  once  and  this  was  only  at  the  lowest  resolution.  Our  very 
first  initial  guess  at  the  lowest  resolution  is  all  zeros  for  the  three  displacements  and  three 
angles.  Based  on  our  experience,  we  set  the  CC  thresholds  at  0.65,  0.70  and  0.75,  and  the 
maximum  numbers  of  restarts  at  20, 10  and  5,  from  low  to  high  resolutions,  respectively. 

3.  Evaluation  of  registration 

3.1.  Registration  accuracy  based  on  bony  landmarks 

We  evaluated  registration  of  the  pelvis  by  measuring  the  displacement  of  bony  landmarks 
following  registration.  We  used  six  easily  found  bony  landmarks  consisting  of  two  great 
sciatic  notches,  two  lesser  sciatic  notches,  the  pubic  symphysis,  and  the  coccyx,  some  of 
which  are  illustrated  in  figure  1.  Previously,  sciatic  notches  and  the  pubic  symphysis  were 
used  to  register  CT  and  MRI  images  for  prostate  conformal  radiation  (KagawacJ  al  1997). 
To  measure  landmark  displacements,  we  used  RegViz ,  a  program  written  in  IDL  (Interactive 
Data  Language,  Research  System  Inc.,  USA)  and  created  in  our  laboratory  for  visualizing 
and  analysing  registered  image  volumes.  We  navigated  transverse,  coronal  and  sagittal  MR 
images  slice-by-slice  to  search  the  landmarks.  The  same  unique  features  such  as  corners  and 
intersections  were  identified  with  a  cursor  on  magnified  images.  A  single  person  repeated  this 
six  times  over  a  few  weeks  and  results  were  averaged  to  give  a  3D  location  for  each  landmark. 
A  radiologist  confirmed  the  landmark  selection.  Following  registration,  we  calculated  the 
root-mean-squared  (RMS)  distance  over  the  six  landmarks  (Wang  et  al  1996). 

Although  this  method  provides  an  independent  means  for  evaluating  skeletal  registration 
accuracy,  there  is  error  in  localizing  the  bony  landmarks.  To  determine  the  effect  of  localization 
error,  we  performed  least-squares  point-to-point  registration  (Maurer  et  al  1997)  and  compared 
results  to  MI  registration.  The  rationale  is  that  if  we  could  identify  point  landmarks 
without  error  on  the  bony  pelvis,  point-to-point  registration  would  be  perfect.  Hence, 
any  displacement  left  after  registration,  is  introduced  by  localization  error.  We  determined 
the  optimal  transformation  for  matching  the  six  corresponding  landmarks.  Points  were 
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Figure  1.  MR  prostate  image  with  labelled  features  used  to  analyse  registration  error.  This 
transverse  image  is  from  the  reference  volume  of  prostate  cancer  patient  P3.  The  prostate  boundary 
was  manually  segmented  near  the  image  centre.  The  four  vertical  dash  lines  from  left  to  right 
indicate  the  rim  of  the  right  acetabular  socket,  the  right  and  left  rims  of  the  prostate  and  the  rim  of 
the  left  acetabular  socket,  respectively.  The  five  crosses  from  bottom  to  top  indicate  the  coccyx, 
the  prostate  posterior  rim,  the  2D  centroid  automatically  calculated  from  the  segmented  prostate 
area,  the  anterior  rim  and  the  pubic  symphysis.  The  image  also  shows  the  bottom  of  the  bladder, 
the  rectum,  the  pubic  symphysis  and  hip  joints. 


transformed  and  distances  between  corresponding  points  were  determined.  RMS  values 
were  computed  and  compared  to  the  RMS  values  from  MI  registration. 

3.2.  Registration  consistency 

We  calculated  the  registration  consistency  as  proposed  by  Freeborough  et  al  (1996).  For  each 
of  the  three  volunteers,  we  used  three  volumes:  reference,  diagnosis  and  empty  bladder,  all 
of  which  were  obtained  with  the  subject  in  the  supine  position.  We  call  these  three  volumes 
A,  B  and  C,  respectively.  They  give  three  pairs  of  registrations  (A-B,  B-C  and  C-A)  and 
three  sets  of  transformation  parameters  ( Tah ,  T ^  Tca).  Using  the  transformation  parameters, 
we  transformed  voxel  positions  in  A  to  B,  and  then  to  C,  and  then  back  to  A.  The  distance 
between  the  original  location  and  the  final  position  is  calculated.  Since  this  is  introduced  by 
three  transformations,  we  estimate  the  error  for  a  single  transformation,  by  multiplying  by 
3-1/2  (Freeborough  et  al  1996). 

3.3.  Voxel  displacements 

To  test  the  dependency  of  registration  on  algorithmic  features  such  as  image  cropping,  one 
can  compare  transformation  parameters.  However,  we  chose  a  more  meaningful  approach 
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that  consisted  of  finding  the  average  displacement  of  voxels  in  a  region-of-interest  (ROI) 
(Carrillo  et  al  2000).  The  3D  distances  between  transformed  voxels  were  calculated  in 
millimetres  and  averaged  over  a  cubic  ROI  just  covering  the  prostate. 

3.4.  Other  evaluation  methods  including  displacement  of  prostate  centroids 

We  used  a  variety  of  other  methods  to  evaluate  the  registration  of  the  pelvis  and  prostate.  First, 
we  measured  potential  displacements  of  the  3D  centroid  of  manually  segmented  prostates. 
Second,  we  used  multiple  visualization  and  analysis  methods  found  in  RegViz  and  MIM™ 
(Zalen  LLC,  Novelty,  OH  44072)  such  as  contour  overlap  and  colour  overlay.  Third,  we 
calculated  the  intensity  difference  between  the  reference  and  registered  volumes  on  a  voxel- 
by-voxel  basis  and  computed  statistics.  Fourth,  we  calculated  the  correlation  coefficient 
between  corresponding  voxels  to  measure  the  quality  of  registration  of  two  MR  volumes 
acquired  with  identical  parameters. 

4.  Results 

4.1.  Comparison  of  mutual  information  and  correlation  coefficient 

In  figure  2,  we  compare  the  two  similarity  measures  at  different  resolutions.  Plotted  are  MI 
and  CC  values  as  a  function  of  translation  along  the  transverse  axis  where  the  origin  is  the 
optimal  transformation.  For  images  at  a  resolution  of  1  /4  voxels  along  a  linear  dimension,  the 
CC  curves  are  much  smoother  than  MI,  which  is  noisy  and  contains  many  local  maxi  mums 
as  shown  in  figure  2(a).  In  addition,  there  is  a  false  global  maximum  in  Figure  2(a)  at  18 
voxels.  At  full  resolution,  figure  2(c)  shows  that  MI  is  much  more  peaked  than  CC,  but  there  is 
high-frequency  noise  in  the  MI  curves  far  from  the  optimum  that  gives  rise  to  local  maximums 
that  must  be  avoided.  From  these  figures,  we  infer  that  CC  is  better  at  low  resolution  and 
that  MI  is  better  at  full  resolution  when  one  is  close  to  the  optimum  value.  As  described  in 
section  2,  our  registration  algorithm  makes  use  of  these  features. 

42.  Assessments  of  pelvic  registration 

Following  registration,  we  determined  displacements  between  the  six  bony  landmarks.  For 
each  subject,  there  was  no  consistent  displacement  of  landmarks  in  one  direction  versus 
another.  Hence,  we  measured  3D  distances  and  determined  RMS  values  over  the  six  landmarks. 
Registration  results  are  plotted  in  figure  3.  The  smallest  errors  are  obtained  when  subjects 
are  in  the  diagnostic  position  for  both  imaging  sessions,  labelled  diagnosis-reference.  The 
average  error  across  the  three  patients  and  three  volunteers  is  only  1 .6  ±  0.2  mm.  Consistently 
larger  errors  are  obtained  when  we  compare  volumes  acquired  in  the  treatment  position  with 
those  in  the  reference  position.  Even  though  the  MR  acquisition  technique  used  for  the  patients 
gave  inferior  image  quality  as  compared  to  that  for  the  volunteers,  the  errors  were  small. 

Additional  error  analyses  are  performed  on  the  volunteer  images  to  assess  the  accuracy  of 
point  landmark  localization.  We  used  images  obtained  with  the  rapid-gradient  echo  sequence, 
which  have  improved  the  contrast  between  the  prostate  and  bony  landmarks  and  which  give  us 
more  confidence  in  measurements.  The  isotropic  voxels  are  1.4  mm  on  a  side,  almost  as  large 
as  the  1.7  ±  0.5  mm  error  obtained  for  the  volunteer  diagnosis-reference  data.  We  assess  the 
error  in  localizing  the  bony  landmarks  by  performing  point-based  registration  on  4-6  points 
per  volume  pair.  The  RMS  distances  after  registration  averaged  across  the  three  volunteers 
was  1.5  db  0.2  mm,  very  nearly  the  value  obtained  with  MI  registration.  Hence,  the  ‘error’ 
reported  for  MI  is  probably  overestimated  due  to  landmark  location  error.  This  analysis  was 
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Figure  2.  MI  and  CC  similarity  functions  are  plotted  to  show  their  relative  advantages  for 
registration  at  different  resolutions.  Two  high-resolution  MRI  volumes  were  registered  to  obtain 
the  optimal  parameters.  We  then  computed  similarity  values  as  a  function  of  translation  along 
the  transverse  axis.  MI  is  plotted  in  (a)  and  (c);  CC  is  plotted  in  (b)  and  (d).  Graphs  on  the  top 
(a)  and  (b)  are  at  a  resolution  of  1  /4  voxels  along  a  linear  dimension,  giving  a  distance  between 
voxel  centres  of  « 5.5  mm.  MI  gives  a  noisy  plot  having  many  local  maximums  and  a  false  global 
maximum  occurs  at  18  voxels.  Graphs  at  the  bottom  are  obtained  at  full  resolution.  MI  has  a 
much  sharper  peak  than  CC,  which  is  relatively  flat.  The  voxel  size  is  1.4  mm.  Images  are  from 
volunteer  V2  in  the  diagnostic  and  reference  conditions. 


prompted  by  ideas  in  a  previous  report  (Maurer  et  al  1997)  that  numerically  demonstrated  the 
relationship  between  point  localization  uncertainty  and  point-based  registration  uncertainty. 

Figure  4  shows  image  intensity  differences  between  reference  and  registered  volumes. 
The  means  are  quite  small  with  8  out  of  10  registrations  giving  a  mean  absolute  value  <  1 .5  grey 
levels,  or  only  1.7%  of  typical  mean  values  of  90  grey  levels  for  these  3D  MR  acquisitions. 
Again,  the  only  consistent  outliers  occur  when  we  compare  the  treatment  position  to  the 
reference.  For  the  case  diagnosis-reference,  extremely  small  image  differences  are  found 
with  VI  giving  0.1  dt  1.6  grey  values,  a  standard  deviation  that  compares  favourably  with  the 
expectation  from  image  noise  alone,  or  1 .5  grey  values.  For  this  volume  pair,  the  subtracted 
images  have  very  little  structure  except  at  the  skin  surface,  indicating  excellent  registration 
(not  shown).  We  know  that  3D  alignment  is  achieved  because  all  slices  across  the  entire  pelvis 
are  well  aligned  and  because  rendered  images  show  that  the  prostate  matches  well. 
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-  Reference 

Figure  3.  Registration  error  as  determined  from  bony  landmarks.  RMS  distances  as  well  as 
maximums  and  minimums  are  plotted  that  show  the  spread  of  the  data.  Conditions  along  the 
jt-axis  such  as  the  ‘treatment’  position  are  described  in  section  2.  P’s  and  V’s  refer  to  patients  and 
volunteers,  respectively.  Averaging  data  across  all  subjects  for  the  best  case  (diagnosis-reference) 
gives  1.6  ±  0.2  mm.  Averages  are  2.9  ±  0.7  mm  and  2.0  ±0.1  mm  for  treatment-reference  and 
empty  bladder-reference  respectively. 


Figure  4.  Image  intensity  difference  between  registered  volume  pairs.  Means  and  standard 
deviations  calculated  over  the  entire  volumes  are  plotted.  Other  details  are  given  in  the  legend  of 
figure  3.  Eight  out  of  ten  registrations  have  a  mean  absolute  grey  level  difference  less  than  1.5  grey 
levels.  Average  results  are  0.9  ±  1.8,  1.5  ±  3.9,  and  0.4  ±  2.6  grey  levels,  or  1.0,  1.7  and  0.4% 
of  typical  mean  values  of  90  grey  levels,  for  diagnosis-reference,  treatment-reference  and  empty 
bladder-reference  respectively. 
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Registration  consistency,  as  described  in  section  3,  provides  yet  another  means  of 
evaluating  the  quality  of  registration.  Values  were  0.4, 0.8  and  0.7  mm  for  volunteers  V 1 ,  V2, 
and  V3,  respectively.  The  average  is  0,6  ±  0.2  mm,  a  value  less  than  half  the  dimension  of  a 
voxel  indicating  excellent  registration  consistency. 


43.  Assessment  of  prostate  registration 

We  determined  the  quality  of  prostate  registration  by  visually  examining  nearly  all  of  the 
roughly  800  registered  image  slices  using  one  or  more  of  the  methods  found  in  Reg  Viz  and 
MIM™.  A  typical  example  for  the  case  of  diagnosis-reference  is  shown  in  figure  5  where  the 
boundary  overlap  is  excellent  and  probably  within  the  manual  segmentation  error.  In  some 
other  cases  such  as  treatment-reference,  small  displacements  of  the  prostate  were  observed. 
In  a  typical  volume  pair,  the  prostate  is  displaced  to  the  posterior  direction  by  ^3.0  mm  when 
the  legs  are  raised.  There  are  no  obvious  displacements  in  other  directions. 

Centroid  vector  displacements  can  also  be  analysed  following  registration.  For  the  case  of 
diagnosis-reference,  centroid  displacements  are  only  1 .4  ±  0.2  mm.  In  the  case  of  treatment- 
reference,  there  is  a  consistent  displacement  (^3  mm)  in  the  posterior  direction  with  relatively 
little  change  in  the  two  orthogonal  directions.  In  the  case  of  empty  bladder-reference,  two 
of  the  three  volume  pairs  show  a  displacement  in  the  posterior  direction  while  the  other  is 
displaced  in  the  anterior  direction.  Finally,  in  the  case  of  a  diagnostic  volume  obtained  1  week 
before  the  reference,  there  was  a  4  mm  displacement  in  the  caudal  direction  due  mostly  to 
changes  in  rectal  and  bladder  filling.  Because  the  3D  centroid  of  the  prostate  averages  over  a 
large  region,  we  believe  these  measurements  to  be  relatively  insensitive  to  segmentation  error. 
Even  so,  we  consider  the  uncertainty  to  be  at  least  1  mm,  and  displacements  less  than  this 
should  be  disregarded.  All  significant  results  above  can  be  visually  confirmed. 

Prostate  volumes  were  measured  for  each  subject.  The  typical  difference  between  volumes 
in  an  imaging  session  was  <1.5%,  indicating  that  segmentation  errors  were  small  and  that 
prostate  volumes  did  not  change.  The  average  prostate  volume  for  the  healthy  volunteers  was 
23.9  =t  3.2  cm3.  Volume  measurements  are  particularly  useful  for  clinicians  when  assessing 
the  response  of  prostate  cancer  treatments  such  as  brachytherapy,  chemo-  or  radiation  therapy. 


4.4.  Effects  of  image  cropping 

In  figure  6,  we  plot  registration  error  as  determined  from  bony  landmarks  with  and  without  the 
cropping  operation  described  in  section  2.  For  the  case  treatment*- reference,  cropping  always 
improved  registration  accuracy,  and  for  V3,  error  reduced  greatly  from  12.6  mm  to  3.4  mm. 
For  all  other  cases,  subjects  were  always  in  the  supine  position  with  legs  flat  on  the  table, 
and  there  was  no  consistent  effect  of  cropping.  If  anything,  cropping  tends  to  increase  error 
in  these  cases,  with  an  increase  in  five  of  seven  volume  pairs.  Correlation  coefficient  always 
improves  with  image  cropping. 

Displacements  of  bony  landmarks  might  significantly  overestimate  the  change  near  the 
prostate.  Hence,  as  described  in  section  3,  we  investigated  the  displacement  of  voxels  in  a 
ROI  surrounding  the  prostate  between  registrations  with  and  without  cropping.  For  nine  of  ten 
analysed  volume  pairs,  the  average  voxel  displacement  was  <0.5  mm  indicating  that  prostate 
registration  is  fairly  insensitive  to  cropping.  However,  for  V3  treatment-reference,  a  much 
larger  voxel  displacement  of  7.4  mm  was  obtained  indicating  that  cropping  is  critical  for  this 
volume  pair. 
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Figure  6.  The  effect  of  image  cropping  on  registration  accuracy.  The  light  and  dark  bars  are  RMS 
distances  between  bony  landmarks  with  and  without  image  cropping,  respectively,  as  defined  in 
section  2.  Conditions  on  the  jr-axis  are  described  in  section  2. 


4.5 .  Implementation  issues 

The  algorithm  was  quite  robust  and  always  gave  very  nearly  the  same  transformation 
parameters  (less  than  0.01  voxels  and  0.01°)  for  the  22  volume  pairs  in  this  study  using  a 
wide  variety  of  initial  guesses.  The  restarting  and  multi-resolution  features  are  important  and 
we  report  some  results  for  a  typical  volume  pair  registration.  The  multi-resolution  approach 
enabled  the  program  to  get  close  to  the  final  value  quickly  because  of  the  reduced  number  of 
calculations.  That  is,  the  time  for  reformatting  at  the  lowest  resolution  (1/4)  was  9.8  s,  which 
was  less  than  1/59  times  that  at  the  highest  resolution,  a  value  nearly  equal  to  1  /64  expected 
from  the  change  in  the  number  of  voxels.  The  number  of  restarts  was  5, 1  and  1  for  resolutions 
at  1  /4,  1  /2  and  the  full  number  of  voxels.  Each  call  to  the  Simplex  optimization  resulted  in 
55  to  94  MI  evaluations  before  the  tolerance  (0.001)  was  reached.  The  simplex  optimization 
method  worked  about  1 .5-2.0  times  faster  than  the  Powell  method  in  our  implementation.  The 
time  for  registration  using  Simplex,  typically  10  mines  on  a  Pentium  IV,  1.7  GHz  CPU,  with 
1  Gbytes  of  memory,  could  probably  be  greatly  improved  with  optimized  C  code  rather  than 
IDL. 


5.  Discussion 

5.7.  Registration  accuracy 

Our  results  suggest  that  MI  can  be  used  to  accurately  register,  with  an  error  on  the  order  of  a 
voxel,  MR  pelvic  images  obtained  under  similar  conditions.  Because  it  gives  an  independent, 
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true  3D  measurement,  we  like  to  use  the  method  of  point  bony  landmarks  to  assess  accuracy. 
However,  as  argued  in  section  4.2,  the  true  MI  registration  accuracy  might  be  better  than 
our  ability  to  measure  it  with  point  bony  landmarks.  That  is,  following  point  landmark 
registration,  the  distance  between  registered,  corresponding  landmarks  was  on  the  order  of 
that  following  MI  registration.  Very  possibly,  MI  is  more  accurate  than  point  registration 
using  bony  landmarks.  Additional,  independent  evidence  of  excellent  MI  accuracy  comes 
from  the  very  low  error  value  from  the  registration  consistency  measurement  (0.6  ±  0.2  mm). 
Interestingly,  this  is  obtained  even  though  the  interpolation  artifact  present  in  MI  similarity 
surfaces  should  reduce  the  likelihood  of  sub- voxel  accuracy  (Pluim  et  al  2000a).  Our  results 
for  the  pelvis  with  image  volumes  obtained  under  the  same  conditions  compare  favourably 
with  those  for  the  brain,  where  MI  registers  images  very  accurately  giving  errors  as  small  as 
0.7-0. 8  mm  for  CT-MR  (West  et  al  1997). 

Visual  and  quantitative  evaluation  of  prostate  organ  movement  showed  good  registration 
even  when  we  acquired  images  under  conditions  that  greatly  stressed  the  ability  to  register  the 
images.  The  small  prostate  displacements  in  our  study  are  consistent  with  earlier  reports  on 
respiration-induced  prostate  movement  of  <1  mm  for  most  patients  in  supine  position  with 
‘quiet’  respiration  (figure  3(b)  of  the  report  by  Malone  et  al  (2000)).  The  difference  between 
the  treatment  and  diagnostic  positions  resulted  in  the  most  consistent  and  largest  displacement 
of  the  prostate.  When  images  were  acquired  in  the  diagnostic  position  1  week  apart,  there 
was  significant  displacement  of  the  prostate  due  to  a  change  in  rectal  filling.  This  is  consistent 
with  previously  reported  results  (Herk  et  al  1995,  Tenhakenef  a/  1991),  which  found  rectal 
filling  to  be  a  significant  factor  in  prostate  displacement. 

There  are  ways  of  limiting  the  small  displacements  of  the  prostate.  One  obvious  remedy 
is  to  acquire  images  in  the  same  position.  That  is,  if  we  want  to  register  an  image  with  that 
obtained  in  the  treatment  position,  we  should  obtain  it  in  the  treatment  position.  Although  it 
is  unknown  how  accurately  one  must  repeat  the  treatment  position,  a  device  to  support  and 
constrain  the  legs  is  probably  required.  In  addition,  there  is  a  dependence  of  registration  error 
on  bladder  and  rectum  content.  One  solution  is  use  clinical  preparations  often  employed  to 
void  the  bladder  and  rectum  prior  to  prostate  imaging  or  therapy.  We  anticipate  that  this  might 
even  lessen  prostate  displacements  between  the  diagnostic  and  treatment  positions. 

We  must  consider  our  results  with  regard  to  potential  applications  such  as  those  described 
in  the  section  1 .  First,  registered  images  acquired  before  and  immediately  after  treatment  can 
be  used  to  determine  whether  a  tumour  is  adequately  treated.  Second,  serial  examinations  can 
be  registered  to  determine  tumour  progression  or  regression.  Third,  registration  of  functional 
images  from  other  modalities  such  as  nuclear  medicine  or  from  MR  spectroscopy  can  give 
molecular  markers  for  prostate  cancer  (Lee  et  al  2001a,  2001b).  Fourth,  we  want  to  register 
high-quality  MR  images  with  a  few  live-time  interventional  MR  images  to  aid  treatment 
decisions  (Fei  et  al  2001).  Our  results  indicate  that  registering  images  from  the  treatment  and 
diagnostic  positions  can  lead  to  errors  and  potential  steps  are  described  above  to  limit  this 
error.  With  images  acquired  in  the  same  position,  our  results  place  a  lower  limit  on  registration 
error  of  about  1  voxel.  Experiments  are  being  conducted  to  see  if  this  can  be  achieved  with 
interventional  MR  images  (Fei  et  al  2001). 


5.2.  Assessment  of  registration 

We  are  involved  in  a  long-term  effort  to  use  registration  for  detection,  assessment  and  therapy 
of  prostate  cancer.  Hence,  we  have  developed  and  used  several  methods  to  assess  pelvic  and 
prostate  registration. 
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It  is  highly  desirable  to  have  an  automatic  method  for  evaluating  the  quality  of  a  registration 
so  that  a  poor  one  can  be  flagged  before  it  is  used  clinically.  The  correlation  coefficient  would 
be  applicable  whenever  one  uses  MR  images  obtained  with  identical  pulse  sequences.  It 
compares  favourably  with  the  bony  landmark  results.  Registration  consistency  provides  an 
additional  means  to  evaluate  registration  accuracy  that  does  not  rely  on  operator  interaction. 

Other  evaluation  methods  are  applicable  for  clinical  or  research  applications.  RegViz 
and  MIM™  provided  visual  inspection  tools  for  quick  evaluation  of  the  quality  of  registration 
and  potential  prostate  displacement.  Such  methods  can  be  used  to  verify  the  quality  of 
registration  and  possibly  account  for  small  displacements  in  some  applications.  Boundary 
overlays  provide  a  good  means  to  evaluate  organ  deformation  as  well  as  displacement.  Point 
anatomical  landmarks  provide  a  useful,  independent  test;  but  it  is  time  consuming  to  identify 
them;  and  MI  might  be  more  accurate  than  the  point  landmarks.  Centroids  are  obtained 
reliably  because  small  segmentation  errors  are  removed  by  integrating  over  the  entire  prostate 
volume.  Centroids  provide  a  good  means  of  quantifying  prostate  displacements. 


5J.  Algorithm  with  combined  similarity  measures 

Using  both  CC  and  MI  at  different  resolutions  was  an  important  feature  that  increased 
robustness.  When  only  mutual  information  was  used,  registrations  at  low  resolution  sometimes 
gave  false  solutions  that  misled  registration  at  the  next  higher  resolution.  However,  CC 
performed  well  and  gave  many  fewer  local  maximums  at  the  lower  resolutions  (figure  2(a) 
and  (b)).  But  MI  gave  a  more  accurate  solution  at  the  full  resolution  due  to  the  peaked  MI 
surface  (figure  2(c)  and  2(d)).  Our  registration  algorithm  combined  advantages  from  the  two 
similarity  measures. 

There  are  probably  several  reasons  why  mutual  information  does  not  work  well  at  low 
resolution.  First,  the  similarity  curve  is  noisy  with  periodic  oscillations  from  the  so-called 
interpolation  artifact  (Pluim  et  al  2000a)  that  is  accentuated  at  reduced  resolutions  (Pluim  et  al 
2000b).  This  results  in  the  many  local  maximums  in  figure  2(a)  that  can  trap  the  optimization. 
A  similar  result  was  reported  for  brain  registration  (Lau  et  al  2001,  Maes  et  al  1997).  Second, 
when  images  are  of  low  resolution  and  there  is  only  a  small  region  of  overlap,  the  mutual 
information  function  can  even  contain  incorrect  global  maximmums  (Pluim  et  al  2000b). 
Such  a  result  was  found  in  figure  2(a)  where  the  global  maximum  was  obtained  at  very  large 
displacements  where  the  overlap  was  reduced.  This  occurs  because  MI  is  not  only  a  function 
of  how  well  the  images  match  in  the  overlap,  but  also  by  how  much  information  is  provided 
by  the  two  images  in  the  overlap  (Studholme  et  al  1997). 


5.4.  Computer  implementation 

Accuracy  is  an  important  issue  for  automatic  registration,  but  there  are  others  such  as 
robustness,  speed,  and  requirements  for  operator  interaction.  With  the  multi-resolution  and 
restarting  features,  our  modified  MI  algorithm  is  quite  robust.  For  a  wide  range  of  initial 
guesses,  it  worked  well  for  all  22  volume  pairs  reported  here.  Three  of  the  volume  pairs  were 
from  patients  and  we  are  confident  that  routinely  acquired  clinical  images  will  have  sufficient 
quality  for  registration.  Because  good  starting  values  are  unimportant,  operator  interaction  is 
minimal.  In  one  instance,  cropping  of  the  legs  was  important  for  registering  an  image  volume 
obtained  in  the  treatment  position  with  that  in  the  diagnostic  position.  It  is  not  surprising  that 
legs  in  a  very  different  position  have  to  be  cropped.  Although  this  is  easy  to  do  manually,  we 
can  probably  determine  an  automated  method  if  it  is  deemed  desirable. 
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The  mutual  information  similarity  measure  is  quite  robust.  Even  though  our  images  are 
very  similar,  we  had  less  success  with  some  other  measures  such  as  the  sum  of  the  squared 
image  difference.  An  advantage  of  MI  is  that  it  can  be  used  with  images  from  different 
modalities,  a  feature  that  we  are  starting  to  use. 

6.  Conclusion 

We  have  developed  an  automatic  volume  registration  algorithm  with  special  features  for  the 
pelvis  and  prostate  MR  volumes.  When  both  volumes  in  a  pair  were  obtained  in  the  diagnostic 
position  under  comparable  conditions,  our  internal  registration  measures  showed  accuracy  on 
the  order  of  a  voxel.  We  believe  that  the  MR  image  registration  method  is  sufficiently  accurate 
and  robust  for  a  variety  of  applications  of  interest  in  the  pelvis  and  prostate.  We  are  beginning 
to  explore  these  applications  in  clinical  procedures  and  animal  experiments. 
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ABSTRACT 

We  created  and  evaluated  an  almost  fully  automated,  three-dimensional  non-rigid 
registration  algorithm  using  mutual  information  and  a  thin  plate  spline  (TPS) 
transformation  for  MR  images  of  the  prostate  and  pelvis.  In  the  first  step,  an  automatic 
rigid  body  registration  was  used  to  capture  the  global  transformation.  Algorithm  features 
included  a  multi-resolution  approach,  two  similarity  measures,  and  automatic  restarting  to 
avoid  local  minimums.  In  the  second  step,  local  feature  points  were  registered.  An 
operator  entered  only  five  feature  points  (FP’s)  located  at  the  prostate  center,  the  left  and 
right  hip  joints,  and  the  left  and  right  distal  femurs.  The  program  automatically 
determined  other  FP’s  at  the  external  pelvic  skin  surface  and  along  the  femurs.  The 
program  optimized  mutual  information  of  a  small  cubic  volume  of  interest  (VOI) 
centered  on  each  FP  by  varying  three  translations  and  three  rotations.  From  each 
optimized  VOI,  nine  control  points  (CP’s)  were  obtained  at  the  center  and  the  comers  of 
the  VOL  More  than  600  CP’s  were  used  to  establish  a  TPS  transformation  for  the 
deformation  of  the  pelvic  region  and  the  prostate.  Experiments  on  healthy  volunteers 
determined  the  quality  of  registration  under  conditions  found  in  potential  applications  in 
prostate  cancer  treatment  and  interventional  MRI  guided  therapies.  Images  were  acquired 
in  the  diagnostic  (supine)  and  treatment  position  (supine  with  legs  raised).  For  10  volume 
pairs  obtained  in  much  different  positions,  non-rigid  registration  was  much  better  than 
rigid  body  registration.  Using  various  visualization  techniques,  it  was  clear  that  warping 
rectified  the  sometimes  significant  pelvic  misalignment  by  the  rigid  body  method.  Gray 
value  measures  of  registration  quality  including  mutual  information,  correlation 
coefficient,  and  intensity  difference  all  improved  with  warping.  Importantly,  the  prostate 
tended  to  be  more  accurately  registered  with  warping;  the  distance  between  prostate 
volume  centroids  was  only  0.7  ±  0.2  mm  following  warping  as  compared  to 
4.9  ±  3.4  mm  with  rigid  body  registration.  From  visual  and  quantitative  measurements, 
we  found  that  our  almost  fully  automatic  method  worked  as  well  as  a  previously  reported 
method  using  non-rigid  registration  with  manually  placed  control  points.  The  method  was 
computationally  efficient  for  the  600  control  points  typically  used.  We  believe  that  the 
non-rigid  registration  method  will  provide  a  powerful  tool  for  many  applications  in  pelvic 
and  prostate  imaging. 


Key  words:  automatic  non-rigid  image  registration,  mutual  information,  thin  plate  spline, 
interventional  magnetic  resonance  imaging  (iMRI),  prostate  cancer. 
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I.  INTRODUCTION 

We  are  investigating  three-dimensional  (3D)  non-rigid  image  registration  to  be  used 
in  applications  of  prostate  cancer  diagnosis,  staging,  and  therapy.  In  particular,  we  are 
interested  in  applications  related  to  the  minimally  invasive  interventional  MRI  (iMRI) 
guided  treatment  of  prostate  cancer.  At  our  institution,  we  currently  use  iMRI  on  a  low- 
field  open  magnet  system  to  guide  radiofrequency  (RF)  thermal  ablation  of  abdominal 
cancer[l]-[3]  and  we  are  investigating  this  method  for  prostate  cancer  treatment.  A 
unique  feature  of  iMRI-guided  thermal  ablation  is  that  therapy  can  be  monitored  with  MR 
either  by  acquiring  images  of  thermally  induced  lesions  or  by  measuring  temperature.  In 
addition,  MR  imaging  of  the  prostate  is  desirable  because  it  more  accurately  delineates 
the  prostate  than  does  CT  [4]  [5]  which  can  overestimate  prostate  volume  [6]  [7]  and 
ultrasound,  which  has  a  tendency  to  underestimate  the  extent  of  lesions  [8]. 

Several  applications  in  prostate  imaging  require  registration.  First,  comparison  of 
registered  MR  images  acquired  before  and  immediately  after  RF  ablation  can  be  used  to 
determine  whether  a  tumor  is  adequately  treated.  This  is  particularly  helpful  in  instances 
where  the  edematous  response  to  treatment  can  be  confused  with  a  highly  perfused  tumor. 
Second,  other  treatment  methods  such  as  radiation  therapy,  brachytherapy,  and  surgery, 
will  also  be  aided  by  registration  of  images  from  pre-,  intra-,  and  post-therapy  for 
treatment  planning,  guidance,  and  assessment.  Third,  inter-modality  registration  will  be 
useful  because  there  are  emerging  techniques  in  nuclear  medicine  [9]  and  MR 
spectroscopy  [10]  that  promise  to  detect  and  delineate  prostate  tumor.  We  have  an  active 
project  aimed  at  registering  and  fusing  SPECT  monoclonal  antibody  images  with  MR  to 
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aid  interventional  MRI  guided  treatments.  Fourth,  registration  of  serial  examinations  can 
be  used  to  follow  regression/progression  of  tumors. 

There  are  challenges  to  the  pelvis  and  prostate  registration.  First,  pelvic  regions  can 
change  shape  significantly,  unlike  the  brain  to  which  registration  has  been  most  often 
applied.  Different  patient  positions  such  as  legs  up  and  down  can  cause  the  movement 
and  deformation  of  internal  organs.  Second,  the  normal  prostate  is  a  small  organ  that 
when  healthy  measures  only  about  3.8  cm  in  its  widest  dimension  transversely  across  the 
base  [11].  Third,  the  small  prostate  is  located  below  a  much  larger  bladder  that  can 
change  shape  and  size.  The  prostate  might  move  relative  to  the  pelvic  bones  due  to 
changes  in  bladder  and  rectal  filling  [12]  [13].  The  alignment  of  the  pelvic  bones,  a  most 
prominent  anatomical  feature  in  MR  gray-scale  images,  does  not  necessarily  mean  that 
the  prostate  is  aligned. 

Some  reports  describe  methods  for  rigid  body  registration  in  the  pelvis  or  prostate 
[12][14]-[21].  Some  of  these  methods  require  either  segmentation  or  visual  identification 
of  structures.  For  example,  manual  registration  has  been  used  where  an  operator  cues  on 
segmented  vascular  structures  [22]  or  other  anatomical  landmarks  [14]-[16].  Others  have 
used  automated  3D  schemes  that  match  contours  of  bones  and  sometimes  other  structures 
that  are  extracted  using  manual  or  interactive  segmentation  [23]. [18] [19]  Manual 
segmentation  has  also  been  used  to  create  surfaces  for  automatic  registration  [20] [21]. 

We  previously  described  a  gray-scale,  mutual  information,  rigid  body  volume-to- 
volume  (W)  registration  method  for  the  pelvic  and  prostate  MR  images  [24].  For  volume 
pairs  acquired  over  a  short  time  span  with  the  volunteer  in  a  similar  position,  rigid  body 
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registration  accuracy  of  both  the  prostate  centroid  (typically  <  1  mm)  and  bony  landmarks 
(average  1.6  mm)  was  on  the  order  of  a  voxel  (~  1.4  mm).  The  centroid  error  was  slightly 
smaller  because  the  prostate  was  at  the  volume  center  and  rotation  errors  had  less  effect 
on  it.  The  localization  error  in  finding  3D  points  from  bony  landmarks  is  probably  greater 
than  that  of  finding  centroids  of  relatively  large  prostate  volumes  where  segmentation 
errors  average  out.  With  rigid  body  registration,  we  obtained  larger  prostate  centroid 
displacements  (2.8  to  10.0  mm)  when  acquisitions  were  obtained  under  much  different 
conditions  (e.g.,  legs  flat  and  legs  raised)  giving  large  anatomical  deformations.  Rigid 
body  registration  of  the  pelvis  is  inadequate  under  such  conditions  [24]. 

Non-rigid  registration  is  a  solution,  and  there  are  a  number  of  relevant  reports  [25]- 
[33].  However,  these  applications  were  mainly  for  brain  [29][31][33]-[35]  and  breast 
[27][30][36];  far  fewer  applications  have  been  described  for  the  abdomen  [28][32][37] . 
As  for  the  prostate,  a  finite  element  model  was  used  to  deform  the  gland  during 
brachytherapy  treatments  and  this  method  also  requires  manual  segmentation  of  the  gland 
[38]. 

We  recently  reported  a  non-rigid  registration  method  that  used  many  manually 
selected  control  points  [3  9]  [40].  After  automatic  global  rigid  body  registration,  the 
operator  manually  selected  over  180  control  points  at  the  prostate  center,  pelvic  surface 
and  internal  structures.  The  program  automatically  optimized  each  control  point  location 
by  displacing  it  in  the  x,  y,  and  z  directions  with  respect  to  the  reference  volume  until  the 
mutual  information  computed  over  a  small  cube  of  voxels  was  maximized.  The  thin  plate 
spline  (TPS)  transformation  was  then  applied  to  express  the  deformation  of  the  pelvic 
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region  and  the  prostate.  This  interactive  method  was  applied  to  pelvic  MR  images  and 
lung  CT-PET  images  [41].  The  time  required  for  control  point  selection  was  a  limitation. 

In  this  study,  we  build  upon  our  previous  experience  and  develop  an  almost  fully 
automatic  non-rigid  registration  method.  Our  goal  is  to  automate  the  algorithm  to  save 
time  and  labor  without  losing  registration  quality.  In  registration  experiments,  we  use 
high-resolution  MR  image  volumes  of  the  pelvis  and  prostate  in  order  to  determine  any 
uncompensated  deformation  following  registration.  We  use  image  data  that  shows 
considerable  deformation;  e.  g.,  images  acquired  in  the  diagnostic  (supine)  and  treatment 
positions  (supine  with  legs  raised).  We  qualitatively  and  quantitatively  compare  results  of 
the  new  non-rigid  registration  algorithm  to  the  previous,  more  manual  version,  and  to 
rigid  body  registration. 


II.  REGISTRATION  ALGORITHM 
A.  Similarity  Measurements 

We  used  two  similarity  measures,  mutual  information  (MI)  and  correlation  coefficient 
(CC)  to  perform  a  gray-scale  registration.  Suppose  one  volume  R  is  the  reference,  and  the 
other  F  is  floating.  Their  mutual  information  MI(R,  F )  is  given  below  [42]. 

M(R,F)  =  logoff' f) 

rj  PR(r)-pF{f ) 

The  parameters  r  and  /  are  the  intensity  values  of  the  reference  and  floating  volumes, 
respectively;  and  their  value  can  range  from  zero  to  the  maximum  image  intensity.  The 
joint  probability  pRF(r,f) and  the  marginal  probabilities  pR(r)  of  the  reference  image 
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and  pF(f)  of  the  floating  image,  can  be  estimated  from  the  normalized  joint  intensity 
histogram. 

The  correlation  coefficient  is  a  measure  of  the  interdependence  of  two  random 
variables;  it  ranges  from  -1  to  +1,  indicating  perfect  negative  correlation  at  -1,  absence  of 
correlation  at  zero,  and  perfect  positive  correlation  at  +1.  For  the  reference  and  floating 
images,  R  and  F,  their  correlation  coefficient  CC(R,F )  is  given  below  [43]. 


N- 1 


’Z(R(.i)-R)(F(f,-F) 


CC(R,F)  = 


i=0 


N- 1 


N-\ 


^(R(i)-Ry\\^(F(i)-F)7 


i=0 


J  L/=o 


Here  R,  F  denote  the  average  intensities  of  the  reference  and  floating  volumes  and  the 
summation  includes  all  N  voxels  within  the  overlap  of  both  volumes. 


B.  Global  Rigid  Body  Registration  Algorithm  with  Special  Features 

Rigid  body  registration  is  the  first  step,  and  it  captures  the  global  transformation  of 
two  volumes.  We  previously  reported  a  similar  method  includes  special  features  to 
improve  robustness  for  MR  pelvic  images  [24].  We  use  a  multi-resolution  approach  and 
perform  registration  from  low  to  high  resolution.  At  low  resolution,  we  resample  both 
images  at  1/4  or  1/2  number  of  voxels  along  each  linear  dimension,  respectively.  The 
similarity  measure  is  iteratively  optimized  by  varying  the  six  rigid  body  transformation 
parameters  (three  translations  and  three  angles).  We  use  CC  at  the  two  lower  resolutions 
because  it  gives  fewer  local  maximums  [44]-[46],  and  because  it  can  be  calculated  faster 
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than  MI.  We  use  MI  at  full  resolution  because  the  peaked  similarity  function  gives  a  more 
precise  solution  than  CC  [45].  To  avoid  local  maximums,  we  include  a  restarting  feature 
where  registration  is  restarted  with  randomly  perturbed  parameters  obtained  from  a 
uniform  distribution  about  the  initial  transformation  values  at  the  current  resolution.  The 
algorithm  restarts  until  the  absolute  CC  is  above  a  threshold  of  0.5  or  the  maximum 
number  of  restarts  is  reached.  Absolute  CC  is  used  for  the  restart  test  rather  than  MI 
because  CC  has  a  well-defined  range  between  0  and  1,  and  because  CC  has  fewer 
problems  with  local  and  incorrect  global  maximums  for  registrations  at  low  resolution  far 
from  the  optimum  value  [24]  [47]. 

We  record  all  important  results  following  an  optimization  cycle  including  the  CC 
and/or  MI  values,  the  number  of  restarts,  and  the  transformation  parameters.  At  the  end  of 
processing  at  a  lower  resolution,  we  always  select  the  transformation  parameters  having 
the  maximum  CC  value.  We  then  scale  the  translation  parameters  appropriately  and 
assign  the  new  parameters  to  be  initial  values  at  the  next  higher  resolution.  At  the  highest 
resolution,  we  select  the  final  transformation  parameters  to  be  those  with  the  maximum 
MI  value. 

We  now  describe  initial  parameter  values  for  the  algorithm.  We  use  an  initial  guess  of 
all  zeros  for  the  transformation  because  the  patient  is  normally  oriented  approximately  the 
same  way  from  one  scan  to  the  next.  We  set  the  maximum  numbers  of  restarts  at  10,  5, 
and  3,  from  low  to  high  resolution,  respectively. 
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C.  Non-rigid  Registration  by  Optimizing  Control  Points 

Following  the  global  rigid  body  transformation,  we  perform  local  non-rigid 
registration  to  rectify  the  misalignment  of  the  pelvic  region  and  the  prostate.  Below  we 
describe  details  of  the  various  steps  of  the  algorithm  outlined  in  Figure  1. 

C.l.  Semi-Automatic  Detection  of  Feature  Points 

After  global  transformation,  we  manually  selected  five  corresponding  pairs  of  feature 
points  (Figure  2)  from  the  two  volumes.  Locations  are:  the  prostate  center  (PFO),  the  two 
hip  joints  (PF1  and  PF2),  and  the  two  distal  femurs  (PF3  and  PF4).  We  first  selected 
feature  points  PFO,  PF1  and  PF2.  We  then  browsed  images  slice-by-slice  to  locate  PF3 
and  PF4.  We  placed  feature  point  at  the  femur  center.  Because  of  the  subsequent 
optimization  later,  the  point  pairs  do  not  have  to  exactly  lie  on  corresponding  features;  an 
error  of  up  to  15  mm  or  ~  10  voxels  is  well  tolerated.  These  feature  points  are  especially 
useful  when  registering  volumes  obtained  in  the  diagnostic  (supine)  and  treatment  (supine 
with  legs  raised)  positions  [24] [39].  Transverse  slices  are  used  to  select  these  points 
because  they  better  show  displacement  of  the  legs.  Images  from  both  volumes  are 
displayed  side  by  side  to  aid  interpretation. 

Using  these  five  FP’s,  the  program  creates  all  other  FP’s  automatically.  First,  for  each 
volume,  a  line  segment  is  created  between  the  FP’s  at  the  hip  joint  and  the  distal  femur 
(dashed  lines  in  Figure  2).  Each  segment  is  divided  into  8  equal  lengths  to  obtain  7 
additional  FP’s.  Altogether,  we  obtained  9  corresponding  FP  pairs  for  each  femur. 
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Second,  the  prostate  center  is  used  to  create  FP’s  at  the  pelvic  surfaces  (Figure  3a). 
Many  radial  lines  are  drawn  through  the  prostate  center  in  a  two-dimensional  (2D) 
transverse  slice.  The  angle  between  lines,  the  angle  increment,  was  normally  18,  20,  or  30 
degrees.  Note  that  we  exclude  the  vertical  line  because  it  normally  intersects  the  buttock 
groove  and/or  the  penis,  regions  of  relatively  little  interest  that  can  change  considerably 
from  one  acquisition  to  the  next.  The  gray-scale  intensity  of  the  pixels  along  each  line  is 
obtained  (Figure  3b).  The  signal  in  air  is  noise  and  normally  less  than  15  gray  levels  as 
compared  to  a  mean  signal  of  about  90  and  maximum  signal  of  about  500  in  the  tissues. 
The  pelvic  skin  surface  gives  high  intensity  signals,  above  30  gray  levels,  because  of  the 
fat  beneath  the  skin.  We  searched  each  line  for  the  most  distant  point  above  a  threshold,  T 
(Figure  3b)  and  set  this  position  to  be  a  surface  feature  point  (Figure  3a).  We  normally 
choose  a  threshold  of  20  gray  levels.  After  processing  all  lines  in  an  image  slice,  the 
program  repeats  the  procedure  for  4  other  transverse  slices  that  are  selected  with  a  gap  of 
about  10-mm  away  from  each  other.  These  transverse  slices  span  ~  40  mm,  much  of  a 
typical  prostate. 

For  a  typical  registration  with  an  angle  increment  of  18  degrees,  there  are  91  pairs  of 
FP’s:  one  at  the  prostate  center,  18  at  the  femurs,  and  72  at  the  exterior  pelvic  surface. 
Their  3D  coordinates  are  stored.  Because  of  the  subsequent  optimization,  the  FP  pairs  do 
not  have  to  exactly  lie  on  corresponding  features;  an  error  of  up  to  15  mm  or  =  10  voxels 


is  well  tolerated. 
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C.2.  Optimization  of  Corresponding  Feature  Points 

Corresponding  feature  points  are  optimized  using  a  mutual  information  similarity 
measure.  A  small  cubic  volume  of  interest  (VOI)  is  centered  at  each  feature  point.  The 
VOI  can  be  32,  48  or  64  voxels  on  a  side,  and  as  reported  later,  the  selection  of  VOI  size 
depends  on  the  amount  of  warping  required.  In  both  the  reference  and  floating  volumes, 
each  corresponding  feature  point  has  its  own  VOI.  The  one  in  the  reference  volume  is  the 
reference  VOI  that  is  known  and  fixed.  The  corresponding  VOI  in  the  floating  volume  is 
the  floating  VOI.  To  perform  the  optimization,  we  translate  and  rotate  the  floating  VOI 
with  respect  to  the  reference  VOI  and  compute  their  mutual  information  over 
corresponding  voxels.  There  are  six  transformation  parameters  to  optimize:  three 
displacements  in  x,  y,  and  z,  and  three  angles  about  each  of  these  three  axes.  We  used  the 
Nelder-Mead  Simplex  algorithm  [48]  for  optimization,  which  has  previously  been  used  to 
successfully  optimize  non-linear  problems  such  as  mutual  information  registration 
[49][50]  and  other  nonlinear  model  [51]. 

Optimal  transformation  parameters  are  independently  obtained  and  recorded  for  each 
of  the  floating  feature  points.  On  rare  occasion,  one  or  more  of  the  translation  or  rotation 
parameters  of  a  floating  FP  are  larger  than  a  reasonable  value  (e.g.,  40  mm  or  45  degrees), 
and  we  eliminate  it. 

C.3.  Determination  of  Control  Points 

For  each  optimized  feature  point  VOI,  we  create  nine  optimal  control  points  (CP’s). 
As  shown  in  Figure  4,  there  is  a  smaller  cube  about  half  the  dimensions  of  a  bigger  cube 
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that  is  used  to  compute  MI.  The  nine  CP’s  are  the  eight  comers  and  the  center  of  the 
small  cube.  These  CP’s  encode  the  position  and  orientation  of  the  VOI.  For  feature  points 
near  the  MR  image  volume  border,  some  CP’s  may  not  be  in  the  volume,  and  we  exclude 
them.  The  total  number  of  CP's  is  usually  >  600  for  an  angle  increment  of  18  degrees. 

C.4.  TPS  Transformation  Using  Optimal  Points 

The  final  major  step  is  to  obtain  the  warped  volume  from  the  floating  volume.  We 
used  backward  warping  that  includes  two  steps  to  obtain  the  warped  volume  [52].  The 
first  step  is  a  spatial  transformation  that  maps  each  voxel  in  the  target  image  to  its  source 
in  the  source  image  [52].  We  use  the  optimal  coordinates  of  control  points  to  establish  a 
three-dimensional  (3D)  TPS  transformation  [53][54]  between  the  floating  (source)  and 
warped  (target)  image  volumes.  For  example,  a  voxel  with  3D  coordinates  of  (64,  78,  24) 
in  the  target  image  is  mapped  to  its  source  at  (63.2,  76.3,  22.1).  The  second  step  is  the 
computation  of  the  intensity  value  at  each  source  location.  Since  the  position  of  the 
source  can  often  be  real  valued,  for  example,  (63.2,  76.3,  22.1),  we  use  tri-linear 
interpolation  to  obtain  its  intensity  value.  We  then  copy  the  intensity  value  and  assign  it 
to  the  target  voxel  in  the  warped  volume.  Finally,  we  obtain  the  warped  volume  from  the 
floating  image. 

We  summarize  parameter  values  for  the  non-rigid  registration.  The  3D  coordinates  of 
the  five  features  points  are  manually  selected  by  user.  The  angle  increment  is  18  degrees; 
the  intensity  threshold  for  pelvic  surface  detection  is  20  gray  scales;  and  the  VOI  size  is 


32  or  64. 
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D.  Additional  Details 

There  are  several  preprocessing  details.  The  input  MR  volume  is  a  3D  MR  acquisition 
giving  256  x  256  x  128  nearly  isotropic  voxels  over  a  field  of  view  covering  the  whole 
pelvis.  We  create  isotropic  voxels  of  about  1.4  mm  on  a  side  using  tri-linear  interpolation. 
Before  optimization,  we  optionally  bin  the  gray  scale  values  of  two  volumes  to  256;  this 
procedure  helps  the  algorithm  perform  better  for  multi-modality  registrations  such  as 
CT/PET  where  their  gray  scales  are  tremendously  different  [41].  For  both  rigid  body 
registration  and  VOI  optimization,  we  use  tri-linear  interpolation  to  obtain  samples  at 
real-valued  voxel  locations.  Optimization  of  similarity  ends  either  when  the  maximum 
number  of  calculations  is  reached  (typically  500)  or  the  fractional  change  in  the  similarity 
function  is  smaller  than  a  tolerance  (typically  0.001).  The  program  is  written  in  IDL 
(Interactive  Data  Language,  Research  System  Inc.,  Boulder,  CO). 


III.  EXPERIMENTAL  METHODS 

A.  Image  Acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system  (Magnetom 
Symphony,  Siemens  Medical  Systems,  Erlangen,  Germany).  An  8-element  phased  array 
body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a  uniform  sensitivity. 
Typically  two  anterior  and  two  posterior  elements  were  enabled  for  signal  acquisition. 


14 


We  used  two  different  MR  sequences.  First,  we  used  a  3D  RF  spoiled  gradient  echo 
steady  state  pulse  sequence  (FLASH)  with  TR/TE/flip  parameters  of  12/5.0/60  which 
give  256x256x128  voxels  over  a  330x330x256-mm  field  of  view  (FOV)  to  yield 
1.3x1. 3x2. 0-mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices.  The 
acquisition  time  was  5.6  min.  This  sequence  was  good  for  pelvic  imaging  but  was  not 
ideal  for  the  prostate.  It  was  used  for  volunteer  SI.  Second,  we  used  a  3D  rapid  gradient 
echo  sequence  (PSIF)  designed  to  acquire  the  spin-echo  component  of  the  steady  state 
response.  The  spin  echo  component  formed  immediately  prior  to  the  RF  pulse,  and  it  was 
shifted  toward  the  prior  RF  pulse  through  appropriate  gradient  waveform  design.  The 
sequence  with  9.4/5.0/60  (TR/TE/flip)  yielded  160x256x128  voxels  over  a  219x350x192- 
mm  rectangular  FOV  and  1.4x1. 4x1. 5-mm  voxels  oriented  to  give  the  highest  resolution 
for  transverse  slices.  There  was  over  sampling  at  31%  in  the  slice  direction  to  reduce 
aliasing  artifacts.  The  acquisition  time  was  4.3  min.  Most  often,  we  used  the  second 
sequence,  which  gave  excellent  image  contrast  for  the  prostate  and  its  surroundings.  It 
was  used  for  volunteers  S2  and  S3. 


B.  Imaging  Experiments 

We  acquired  3D  MRI  volume  images  from  three  normal  volunteers  under  a  variety  of 
conditions  simulating  anticipated  conditions  in  diagnostic  and  treatment  applications.  In 
the  diagnostic  position,  the  volunteer  laid  supine  throughout  MR  scanning.  In  the 
treatment  position,  the  volunteer  was  supine,  and  his  legs  were  supported  at  30°-60° 
relative  to  the  horizon  and  separated  in  a  “V”  with  an  angle  of  60°-90°  between  two  legs. 
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This  is  similar  to  the  lithotomy  position  used  in  prostate  therapies,  and  it  should  provide 
access  for  needle  insertion  in  brachytherapy  or  RF  thermal  ablation.  For  each  volunteer, 
image  volumes  were  typically  obtained  on  the  same  day  within  an  imaging  session.  The 
coil  array  was  centered  on  the  prostate.  All  images  of  a  volunteer  were  acquired  with  the 
same  MRI  acquisition  parameters  so  as  to  ensure  very  similar  gray  values.  In  total,  there 
are  4,  4,  and  5  volumes  for  each  volunteer,  SI,  S2,  and  S3,  respectively.  For  each 
volunteer,  one  volume  was  in  the  treatment  position;  the  other  was  in  the  diagnostic 
position. 

We  performed  registration  experiments  using  treatment-diagnosis  volume  pairs.  For 
volunteers  SI,  S2,  and  S3,  there  were  3,  3,  and  4  treatment-diagnosis  volume  pairs, 
respectively.  Rigid  body  and  non-rigid  registration  were  applied  to  each  pair. 
Additionally,  we  tested  the  methods  on  10  volume  pairs  obtained  in  the  same  diagnostic 
position. 

C.  Registration  Evaluation 

We  used  multiple  visualization  features  of  RegViz,  a  program  written  in  IDL  and 
created  in  our  laboratory  for  visualizing  and  analyzing  registered  image  volumes.  First, 
color  overlay  displays  were  used  to  evaluate  overlap  of  structures.  One  image  was 
rendered  in  gray  and  the  other  in  the  “hot  -  iron”  color  scheme  available  in  IDL.  To 
visualize  potential  differences,  it  was  quite  useful  to  interactively  change  the  contribution 
of  each  image  using  the  transparency  scale.  Second,  we  used  a  sector  display,  which 
divided  the  reference  and  registered  images  into  rectangular  sectors  and  created  an  output 
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image  by  alternating  sectors  from  the  two  input  images.  Even  subtle  shifts  of  edges  could 
be  clearly  seen.  Third,  we  manually  segmented  prostate  boundaries  in  image  slices  and 
copied  them  to  corresponding  slices  from  the  other  volume.  This  enabled  visual 
determination  of  the  overlap  of  prostate  boundaries  over  the  entire  volume.  We  applied 
the  same  method  to  evaluate  pelvic  registration. 

Correlation  coefficient  and  mutual  information  were  calculated  as  indicators  of 
registration  quality.  Since  volumes  to  be  registered  were  acquired  using  the  same 
acquisition  parameters,  high  absolute  CC  values  were  obtained  when  registration  was 
good  [30].  The  higher  the  MI  values,  the  better  the  two  volumes  are  registered. 

We  compared  the  new  semi-automatic  method  with  a  non-rigid  registration  that  used 
manually  selected  control  points  as  reported  by  us  [39].  Briefly,  in  the  previous  method, 
we  first  performed  a  rigid  body  registration,  then  manually  selected  about  180  control 
points,  automatically  optimized  their  locations,  and  finally  warped  the  volume  using  TPS 
interpolation.  The  principal  change  in  the  current  algorithm  is  the  automation  of  control 
point  selection  after  identification  of  five  feature  points.  A  rigid  body  method  was  also 
previously  described  [24]. 

We  measured  potential  displacements  of  the  3D  centroid  of  the  prostate  to  assess 
prostate  registration  error.  We  used  RegViz  to  manually  segment  the  prostate  across  all 
image  slices  and  calculated  a  3D  centroid. 
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IV.  RESULTS 

A.  Determination  of  Feature  Points 

The  method  for  finding  corresponding  FP’s  was  quite  successful.  After  rigid  body 
registration,  our  visualization  tool  made  it  easy  to  approximately  locate  the  prostate 
center,  hip  joints,  and  the  femurs.  The  automatic  detection  method  for  identifying  the 
external  pelvic  surface  was  quite  reliable.  For  all  volume  pairs,  the  program  correctly 
detected  all  surface  points  using  all  angle  increments  and  a  threshold  of  20  gray  levels. 
Figure  5  shows  surface  FP’s  for  a  typical  treatment-diagnostic  volume  pair.  The  method 
for  finding  FP’s  along  the  femurs  also  worked  in  a  robust  fashion. 

In  over  60  non-rigid  registration  experiments  using  10  treatment-diagnosis  volume 
pairs,  we  found  that  more  FP's  generally  improved  registration  quality.  Figure  6  plots  MI 
values  of  registered  volumes  after  warping  as  a  function  of  angle  increment.  When  the 
angle  increment  is  less  than  20  degrees,  MI  saturates.  As  the  angle  increment  increases, 
less  FP’s  are  used  and  the  MI  value  decreases.  The  downside  of  adding  more  CP's  is  the 
increased  time  for  registration.  We  use  an  angle  increment  of  1 8  degrees  in  subsequent 
experiments. 

Adding  feature  points  along  the  femurs  was  important  for  good  registration  of  internal 
structures.  When  only  surface  FP's  were  used,  the  femurs  and  its  surrounding  structures 
did  not  register  well  (not  shown)  even  with  a  very  large  number  of  surface  points  as 
obtained  with  an  angle  increment  of  9  degrees.  Experiments  showed  that  fewer  feature 
points  along  the  femurs  did  not  satisfactorily  remove  deformations  when  the  legs  were 
raised  in  the  treatment  position. 
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B.  Registration  Quality  of  Non-Rigid  versus  Rigid  Body  Registration 

In  Figure  7,  we  compare  non-rigid  and  rigid  body  registration  for  a  typical  volume  pair 
from  the  treatment  and  diagnostic  positions.  Following  non-rigid  registration,  the  prostate 
boundary  overlap  is  excellent  (Figure  7e)  and  within  manual  segmentation  error  as 
assessed  by  an  experienced  operator.  Using  rigid  body  registration,  there  is  significant 
misalignment  throughout  large  regions  in  the  pelvis  as  shown  in  the  overlap  image 
(Figure  7f),  and  alignment  is  greatly  improved  with  warping  (Figure  7g)  where  the  pelvis 
matched  very  well  even  at  the  outer  surfaces.  Similar  results  were  obtained  in  transverse 
slices  throughout  the  prostate.  Other  visualization  methods  such  as  two-color  overlays 
and  difference  images  (not  presented)  also  show  excellent  matching  of  structures. 

Visual  inspections  were  further  verified  by  quantitative  measurements.  Figure  8  shows 
the  prostate  centroid  displacement  following  rigid  body  and  non-rigid  registration. 
Warping  significantly  decreased  the  displacement,  indicating  much  better  registration  of 
the  prostate.  Following  warping,  the  mean  displacement  was  only  0.7  mm  ±  0.2  mm  cross 
all  volume  pairs.  The  later  number  is  the  standard  deviation.  However,  the  mean  was 
4.9  ±3.4  mm  for  rigid  body  registration. 

A  typical  example  is  the  second  pair  of  volunteer  S2.  Following  rigid  body 
registration,  the  prostate  was  misaligned  with  a  displacement  to  the  posterior  of  about 
13.6  mm  when  in  the  treatment  position  (Figure  7d).  A  prostate  rotation  of  about  14 
degree  along  left-right  axis  was  also  found  following  rigid  body  registration.  Following 
non-rigid  registration,  the  3D  prostates  matched  very  well  and  the  centroid  calculated 
from  segmented  images  displaced  by  only  0.9  mm,  or  0.7  voxels. 
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Additionally,  we  examined  10  volume  pairs  with  both  volumes  acquired  in  the 
diagnostic  position.  In  all  such  cases,  rigid  body  registration  worked  as  well  as  non-rigid 
registration.  There  were  no  noticeable  deformations  in  the  pelvic  region,  and  prostate 
centroids  typically  displaced  less  than  1.0  mm  between  the  two  registered  volumes.  Note 
that  this  was  obtained  even  though  volunteers  always  got  up  from  the  table  and  moved 
around  before  being  imaged  again.  In  what  follows,  we  only  report  results  of  treatment- 
diagnosis  volume  pairs. 

In  Figure  9,  we  compare  rigid  body,  manual  warping,  and  our  new  almost  fully 
automatic  non-rigid  methods.  Correlation  coefficient  (CC)  values  between  registered 
volumes  of  treatment-diagnosis  are  plotted.  Both  semi-automatic  and  manual  non-rigid 
registration  significantly  increased  CC  values  in  every  case,  and  a  paired  two-tailed  t  test 
indicated  a  significant  effect  of  warping  at  p<0.5%.  The  semi-automatic  non-rigid 
registration  worked  better  than  the  manual  one  in  7  out  of  10  cases.  This  is  probably 
because  we  used  more  (~  600)  CP’s  with  the  semi-automated  method  than  the  manual 
method  (~  180).  Recall  that  with  the  manual  method  each  FP  was  optimized  for 
translation  and  not  rotation  and  each  FP  gave  one  CP 


C.  Algorithmic  Implementation 

The  rigid  body  registration  is  robust  as  previously  reported  by  us  [24].  For  10  volume 
pairs  acquired  in  the  same  diagnostic  position,  registration  was  100%  successful  with 
prostate  centroids  typically  displaced  <  1  mm  following  registration.  For  10  treatment- 
diagnosis  volume  pairs,  the  pelvic  region  was  aligned  except  for  local  deformations  and 
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displacements.  The  time  for  rigid  body  registration  was  typically  5  minutes  on  a  Pentium 
IV,  1.8  GHz  CPU,  with  1.0  G  Bytes  of  memory. 

We  report  some  details  on  non-rigid  registration  for  a  typical  volume  pair  from 
volunteer  S2.  The  angle  increment  was  18  degrees  and  the  total  number  of  FP's  was  91. 
After  FP  optimization,  21  FP's  were  rejected,  and  70  FP's  remained  that  produced  630 
CP's.  Excluding  17  invalid  CP's  outside  the  volume  region,  there  were  613  useful  CP's  for 
the  TPS  transformation.  Following  rigid  body  registration,  the  mean  distance  between  the 
reference  and  floating  FP's  was  18.6  ±  12.4  mm,  where  the  latter  number  is  the  standard 
deviation.  The  maximum  distance  was  58.2  mm.  The  maximum  rotation  of  a  VOI  was 
40.3  degrees.  Using  the  same  computer  as  described  above,  for  volumes  with 
256x256x140  voxels,  the  non-rigid  registration  took  about  30  minutes  with  TPS  warping 
occupying  ~  86%  of  the  time.  Very  probably,  the  time  could  be  greatly  improved  with 
optimized  C  code  rather  than  the  high  level  language,  IDL. 

VOI  size  was  an  important  parameter  to  control  the  amount  of  warping.  We 
performed  experiments  to  find  the  optimal  VOI  size  using  sizes  of  16,  32,  64,  and  72 
voxels.  We  quickly  found  that  a  VOI  size  of  16  or  72  voxels  on  a  side  gave  unsatisfactory 
results  because  warping  that  was  either  too  much  or  too  little,  respectively.  A  VOI  size  of 
64  worked  somewhat  better  than  other  VOI  sizes  for  SI  and  S3  because  there  was 
relatively  little  deformation  between  volume  pairs.  However,  a  VOI  size  of  32  voxels  on 
a  side  performed  best  for  volunteer  S2  because  there  was  more  deformation  in  this 


case  (Figure  7). 
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The  method  for  obtaining  CP’s  from  FP’s  satisfactorily  encoded  the  local  position  and 
orientation  for  warping.  As  described  previously,  we  used  the  comers  and  the  center  of  a 
small  cubic  volume  contained  within  the  FP  VOI.  Using  the  comers  of  the  FP  VOI  gave 
unsatisfactory  results  from  visual  inspection  as  well  as  CC  and  MI  values. 


V.  DISCUSSION 

A.  Applicability  of  Non-rigid  Registration 

For  MR  images  of  the  pelvis  and  prostate,  non-rigid  registration  is  desirable  whenever 
images  are  acquired  in  different  positions.  Local  deformations  throughout  the  pelvis  can 
be  corrected,  and,  more  importantly,  the  prostate  can  be  accurately  registered.  However, 
when  images  are  acquired  in  the  same  position  under  comparable  conditions  such  as  the 
case  called  diagnosis-diagnosis,  rigid  body  registration  worked  satisfactorily  [24], 
Similarly,  if  one  were  to  reproduce  the  treatment  position  reasonably  well,  one  could 
probably  again  get  very  good  results  with  rigid  body  registration. 

The  high-resolution  MR  images  provide  a  very  stringent  test  for  warping.  Many 
anatomical  details  are  evident,  and  even  a  small  mismatch  can  clearly  be  seen.  With  a 
sufficient  number  of  control  points,  the  TPS  transformation  excellently  approximated  the 
deformations  of  the  pelvis  and  internal  structures  of  our  MR  images.  Even  when  we 
warped  the  volume  in  the  diagnostic  position  to  one  in  the  treatment  position,  most 
organs  were  closely  aligned,  despite  very  significant  movements. 
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With  non-rigid  registration,  we  have  to  be  concerned  about  potential  warping  errors 
affecting  the  application  of  interest.  For  the  prostate,  we  used  only  one  CP  at  the  prostate 
center  because  we  desired  to  maintain  the  spatial  integrity  of  the  organ  and  to  preserve  the 
tissue  volume.  We  placed  many  CP’s  around  the  pelvic  surface  to  produce  reasonable 
warping. 

The  semi-automatic  method  is  fast  and  reliable.  A  user  only  needs  to  place  5  feature 
points  in  a  procedure  that  is  straightforward  with  our  graphical  user  interface.  All  other 
tasks  are  automated.  Further,  the  localization  error  of  feature  points  up  to  15  mm  or 
=  10  voxels  is  well  tolerated.  As  compared  to  the  manual  non-rigid  method,  the  almost 
fully  automated  method  saved  time  and  labor,  and  gave  registrations  that  were  as  good  as 
or  better  than  the  manual  method.  An  interesting  semiautomatic  method  was  previously 
reported  for  landmark  localization  in  the  brain  [29]  that  would  be  time-consuming  for  use 
in  the  pelvis.  A  user  manually  specified  a  region  of  interest  (ROI),  a  3D  differential 
operator  was  applied  to  obtain  landmark  candidates  within  the  ROI,  and  the  user  selected 
the  most  promising  candidate.  Unlike  the  brain  images,  the  pelvic  volume  needs  hundreds 
of  points  (600  in  our  study)  to  correct  the  huge  deformation  not  seen  in  the  brain. 

The  flexibility  of  the  semi-automatic  method  makes  the  current  software  suitable  for 
non-rigid  registration  in  many  applications  in  addition  to  the  clinical  procedures  described 
in  the  Introduction.  We  believe  that  the  registration  method  can  be  applied  to  other 
modalities  and  inter-subject  images.  In  addition,  the  general  approach  can  probably  be 
modified  for  other  organ  systems. 
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B.  Algorithmic  Robustness  and  Efficiency 

For  rigid  body  registration,  two  principal  design  features  greatly  improved  the 
robustness  of  the  global  registration  [24].  First,  using  both  CC  and  MI  at  different 
resolutions  was  an  important  feature  that  increased  robustness.  CC  gave  fewer  local 
minimums  at  low  resolutions  and  MI  was  more  accurate  at  high  resolution  [45]  [46].  Our 
registration  algorithm  combined  these  advantages  from  the  two  similarity  measures. 
Second,  the  restarting  mechanism  was  important.  Without  restarting,  we  found  that 
registrations  sometimes  failed  in  cases  of  volumes  with  large  mismatches  and  significant 
deformation.  Even  these  cases  resulted  in  a  proper  solution  when  restarting  was 
employed. 

The  non-rigid  registration  algorithm  is  designed  to  be  very  computationally  efficient 
for  TPS  warping  with  hundreds  of  CP's.  We  optimized  each  CP  separately  because  the 
optimization  of  six  parameters  (three  translations  and  three  rotations)  is  simple  and  fast. 
Simultaneous  optimization  of  many  control  points  is  another  approach  [55].  However, 
our  experiments  shows  that  600  control  points  with  1800  free  parameters  are  needed; 
hence,  simultaneous  optimization  would  be  extraordinarily  complex  and  time  consuming. 
We  applied  the  TPS  transformation  once  to  the  final  optimal  CP's;  this  saved 
considerable  time  and  gave  satisfactory  warping.  If  TPS  was  applied  during  each  iteration 
of  the  optimization  process  [55],  the  registration  time  would  be  quite  unacceptable.  If  we 
were  to  use  optimized  C  code  instead  of  IDL,  the  rigid  body  and  non-rigid  registration 


should  be  more  time  efficient. 
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C.  Evaluation  of  Non-rigid  Registration 

Since  there  is  no  standard  method,  we  used  a  variety  of  methods  to  evaluate  non-rigid 
registration  quality.  First,  for  routine  evaluation,  a  color  overlay  is  simple,  fast,  and 
intuitive.  To  better  visualize  the  two  data  sets,  we  interactively  adjust  the  transparency 
scale  of  each  image.  Second,  for  illustration  of  subtle  difference  along  an  edge,  we  use  a 
sector  display  because  it  best  shows  small  shifts  [45].  Third,  for  visual  evaluation  of  a 
specific  organ  such  as  the  prostate,  we  like  to  superimpose  manually  marked  contours 
from  one  image  onto  another  as  shown  in  Figure  7.  This  clearly  shows  any  displacement 
or  deformation.  Fourth,  a  more  quantitative  approach  is  obtained  by  calculating  the 
displacement  of  a  3D  centroid  of  the  segmented  prostate.  Finally,  when  images  have 
comparable  gray  levels,  a  difference  image,  and  statistics  on  the  difference  image, 
provides  yet  another  approach.  A  downside  with  MR  images  is  that  the  inhomogeneity  of 
the  signal  response  and  interpolation  can  introduce  artifacts  in  the  difference  images. 
Since  MR  image  intensity  can  vary  with  different  MR  sequence  parameters  and  the  signal 
response  of  MR  coil,  the  gray  value  statistic  may  have  some  limitations  when  image 
acquisitions  are  not  carefully  repeated.  Similarly,  there  are  other  gray-level  measures  such 
as  the  correlation  coefficient.  Although  the  absolute  value  of  CC,  and  other  gray-scale 
measures,  might  not  predict  registration  quality,  it  is  probably  a  very  good  way  to 
compare  registrations  on  a  single  volume  pair,  as  done  in  Figure  9. 

We  conclude  that  the  semi-automatic  non-rigid  registration  works  better  than  rigid 
body  registration  when  patient  position  is  greatly  changed  between  acquisitions.  It  also 
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compares  favorably  to  non-rigid  registration  with  manual  selection  of  control  points.  We 
believe  it  will  be  a  useful  tool  for  many  applications. 
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FIGURE  LEGENDS 

Figure  1.  Flow  chart  of  the  non-rigid  registration  algorithm.  The  first  four  steps  are  to 
create  features  points  in  both  volumes  following  global  rigid  body  registration.  The  loop 
from  FOR  to  END  is  to  optimize  the  volumes  of  interest  (VOI)  as  centered  at 
corresponding  feature  points  using  mutual  information.  The  optimal  VOI  is  then  used  to 
determine  nine  control  points  that  represent  the  position  and  orientation  of  the  VOL  The 
position  information  of  all  control  points  is  used  to  establish  a  three-dimensional  TPS 
transformation  for  the  entire  volume.  See  text  for  details. 

Figure  2.  Locations  of  the  manually  selected  five  feature  points.  FPo  is  at  the  prostate 
center;  FPi  and  FP2,  the  hip  joints;  and  FP3  and  FP4;  the  distal  femurs.  The  dash  lines 
connecting  FPi  and  FP3,  and  FP2  and  FP4  are  along  the  femurs. 

Figure  3.  Automatic  detection  of  the  pelvic  surfaces.  Image  (a)  is  a  transverse  image  slice 
covering  the  prostate.  The  dash  lines  pass  thought  the  prostate  with  an  equal  angle 
increment.  The  dots  are  at  the  pelvic  exterior  surfaces.  Graph  (b)  plots  the  intensity 
signals  along  one  dash  line  in  (a).  A  threshold  T  is  used  to  determine  the  pelvic  surfaces 
along  this  line. 

Figure  4.  Nine  control  points  for  each  feature  point  and  volume  of  interest  (VOI).  The 
big  cubic  volume  is  the  VOI  that  is  used  to  calculate  mutual  information  and  optimize  the 
feature  point  FP.  The  small  cubic  volume  is  with  the  same  center  and  the  same  orientation 
but  half  size  on  a  side.  The  center  and  the  eight  comers  of  the  smaller  cubic  volume  are 
the  control  points  that  will  be  used  to  establish  thin  plate  spline  transformation. 
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Figure  5.  Feature  points  for  images  acquired  in  the  treatment  and  diagnostic  positions. 
Image  (a)  is  from  the  reference  volume  acquired  in  the  treatment  position  with  legs 
raised.  Image  (b)  is  to  be  warped  and  is  from  the  volume  acquired  in  the  diagnostic 
position  with  legs  flat  on  the  table.  Feature  points  are  located  at  the  prostate  center, 
femurs,  and  pelvic  surfaces.  Each  feature  point  is  located  at  one  voxel  but  displayed  much 
larger  for  improved  visualization.  Volumes  are  of  volunteer  S2. 

Figure  6.  Mutual  information  as  a  function  of  angle  increment.  The  y-axis  was  MI  values 
between  registered  volumes.  The  x-axis  was  the  angle  increments  that  were  used  to  detect 
pelvic  surfaces.  Generally,  MI  values  decreased  when  the  angle  increment  increased 
because  less  FP's  participated  non-rigid  registration.  MI  values  saturate  when  the  angle 
increment  is  smaller  than  20  degree.  The  number  of  FP’s  varied  from  15  to  60  for  angle 
increments  of  107  to  43,  respectively. 

Figure  7.  Comparison  of  non-rigid  and  rigid  body  registration  for  volumes  acquired  in 
the  treatment  and  diagnostic  positions.  Image  (a)  is  from  the  reference  volume  acquired  in 
the  treatment  position,  and  the  prostate  is  manually  segmented.  Images  in  the  left  and 
right  columns  are  from  the  floating  volume  acquired  in  the  diagnostic  position  following 
rigid  body  and  non-rigid  registration,  respectively.  To  show  potential  mismatch,  the  pros¬ 
tate  contour  from  the  reference  in  (a)  is  copied  to  (b)  and  (c)  and  magnified  as  the  dashed 
contours  in  (d)  and  (e).  The  movement  of  the  prostate  to  the  posterior  is  corrected  with 
warping  (e)  but  not  rigid  body  registration  (d).  Pelvic  boundaries  manually  segmented 
from  the  reference  show  significant  misalignment  with  rigid  body  (f)  that  is  greatly 
improved  with  warping  (g).  Images  are  transverse  slices  from  volunteer  S2. 
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Figure  8.  Prostate  centroid  displacement  following  rigid  body  and  non-rigid  registration. 
Ten  treatment-diagnosis  volume  pairs  from  three  volunteers  were  registered  using  rigid 
body  and  non-rigid  registration,  respectively.  Following  rigid  body  registration,  the  mean 
displacement  of  prostate  centroid  between  registered  volumes  was  4.9  ±3.4  mm  cross  all 
volume  pairs.  The  latter  number  is  the  standard  deviation.  Following  warping,  the  mean 
displacement  was  only  0.7  ±  0.2  mm.  Warping  significantly  decreased  the  displacement 
indicating  much  better  registration  of  the  prostate. 

Figure  9.  Comparison  of  rigid  body,  semi-automatic,  and  manual  non-rigid  registration 
for  treatment-diagnosis  volume  pairs.  Correlation  coefficient  following  registration  is 
plotted.  Both  semi-automatic  and  manual  non-rigid  registration  increased  the  correlation 
coefficient  as  compared  to  rigid  body  registration  in  each  case  and  this  effect  was 
significant  as  determined  from  all  data  (p  <  0.5%).  The  semi-automatic  method  is  better 
than  the  manual  non-rigid  registration  with  180  control  points  in  7  out  of  10  trials.  There 
are  a  total  10  volume  pairs  from  three  volunteers  as  described  previously. 


Create  reference  and  floating  volumes  following  global  rigid  body  registration 
Place  five  feature  points  (FP)  at  prostate  center,  hip  joints,  and  femurs  in  both  volumes 
Create  feature  points  along  femur  using  line  segment  approximation 
Create  feature  points  on  skin  surface  using  the  prostate  center  and  radial  lines 
Initialize  the  size  of  volume  of  interest  (VOI)  centered  at  FP’s 
FOR  FP  FROM  0  to  N-l  DO  BEGIN 

Optimize  mutual  information  (MI)  between  the  reference  and  floating  VOI’s 

1.  Transform  the  floating  VOI  using  three  translations  and  three  rotations 

2.  Interpolate  to  get  a  reformatted  VOI 

3.  Calculate  MI  between  the  reference  and  reformatted  VOI’s 

4.  Vary  the  three  translation  and  three  rotation  parameters 

5.  Repeat  steps  1-4  until  meeting  function  tolerance  or  maximum  iteration  number 
Record  the  optimized  FP  coordinates  in  the  floating  volume 

END 

Determine  control  points  (CP)  using  the  feature  points  and  VOI’s 

Calculate  thin  plate  spline  transformation  using  the  reference  and  optimized  CP’s 

Interpolate  the  floating  volume  and  get  a  warped  volume 
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ABSTRACT 

We  are  investigating  imaging  techniques  to  study  the  rapid  biochemical  and  physiological  response  of  tumors  to 
photodynamic  therapy  (PDT).  Positron  emission  tomography  (PET)  can  provide  physiological  and  functional  images 
of  cancers.  While  MRI  can  provide  high  resolution  anatomical  images  and  generate  serial,  noninvasive,  in  vivo 
observations  of  morphological  changes.  In  this  study,  we  investigate  image  registration  methods  to  combine  MRI  and 
micro-PET  (pPET)  images  for  improved  tumor  monitoring.  We  acquired  high  resolution  MR  and  PET  18F- 
fluorodeoxyglucose  (FDG)  images  from  mice  with  RIF-1  tumors.  We  used  rigid  body  registration  with  three 
translations  and  three  angular  variables.  We  used  normalized  mutual  information  as  the  similarity  measure.  To  assess 
the  quality  of  registration,  we  performed  slice  by  slice  review  of  both  image  volumes,  manually  segmented  feature 
organs  such  as  the  left  and  right  kidneys  and  the  bladder  in  each  slice,  and  computed  the  distance  between 
corresponding  centroids  of  the  organs.  We  also  used  visual  inspection  techniques  such  as  color  overlay  displays.  Over 
40  volume  registration  experiments  were  performed  with  MR  and  pPET  images  acquired  from  three  C3H  mice.  The 
color  overlays  showed  that  the  MR  images  and  the  PET  images  matched  well.  The  distance  between  corresponding 
centroids  of  organs  was  1.5  ±  0.4  mm  which  is  about  2  pixels  of  pPET.  In  conclusion,  registration  of  high  resolution 
MR  and  pPET  images  of  mice  may  be  useful  to  combine  anatomical  and  functional  information  that  could  be  used  for 
the  potential  application  in  photodynamic  therapy. 

Keywords:  Image  registration,  mutual  information,  magnetic  resonance  imaging  (MRI),  positron  emission 
tomography  (PET),  photodynamic  therapy  (PDT),  micro-PET  (pPET),  cancers. 


1.  INTRODUCTION 

Photodynamic  therapy  (PDT)  is  USFDA  approved  (with  the  photosensitizer  Photofrin)  for  advanced  esophageal,  early 
lung  and  late  lung  cancer.1,2  In  PDT,  a  tumor-localized  photosensitizer  is  irradiated  with  visible  light  to  generate 
reactive  oxygen  that  efficiently  kills  cells  and  ablates  tumors.  PDT  is  a  three-component  treatment.1,2  It  requires  (a)  a 
photosensitizer,  often  a  porphyrin-related  macrocycle,  that  tends  to  accumulate  in  tumors;  (b)  non-thermal  visible  light 
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of  a  wavelength  absorbed  by  the  photosensitizer  and  generally  in  the  red  region  of  the  spectrum  (since  longer 
wavelengths  penetrate  human  tissue  best);  and  (c)  molecular  oxygen.  With  an  adequate  oxygen  supply  and  light 
intensity,  the  site  of  photodamage  depends  on  the  location  of  the  photosensitizer.  An  important  advantage  of  PDT  is 
that  both  the  photosensitizer  and  the  light  are  inert  by  themselves,  and  the  light  can  be  precisely  focused  onto  a 
selected  region,  allowing  extreme  specificity  in  the  localization  of  the  photodynamic  effect.  Consequently,  systemic 
toxicities  are  minimized. 

Imaging  techniques  are  used  for  the  study  of  cancer.  Positron  emission  tomography  (PET)  is  widely  used  to  study 
the  physiology  of  many  types  of  cancer.3’4  Since  tissue  uptake  of  18F-fluorodeoxyglucose  (FDG)  depends  on  both 
blood  flow  and  hexokinase  activity,  it  can  be  a  good  probe  to  investigate  the  mechanisms  of  PDT  that  dependent  on 
the  two  parameters.  However,  because  FDG  uptake  depends  on  both,  it  is  difficult  to  differentiate  on  the  basis  of  FDG 
alone.  We  had  planned  to  also  do  water  (perfusion)  and  carbon  monoxide  (blood  volume).  Lapointe  et  al.  reported  the 
study  of  PET  FDG  imaging  for  monitoring  the  response  of  tumors  to  PDT.3  A  micro-PET  (pPET)  imager  was 
designed  and  built  to  perform  dynamic  in  vivo  PET  scans  of  tumor-bearing  mice  after  PDT.  Reductions  in  FDG 
uptake  of  a  treated  tumor  relative  to  an  untreated  tumor  in  the  same  animal  were  observed.3  Magnetic  resonance 
imaging  (MRI)  has  also  been  used  to  evaluate  PDT-induced  hemorrhagic  necrosis  in  the  murine  Ml  tumor  within  72 
hours  of  treatment  of  male  DBA/2  mice.5'7 

In  this  project,  we  investigate  using  both  pPET  and  high-resolution  MR  imaging  techniques  for  improved  tumor 
monitoring.  Though  PET  images  can  provide  the  dynamic  functional  information,  MR  has  superior  anatomical 
information  for  the  locations  of  tumors  and  organs  and  can  make  serial,  noninvasive,  in  vivo  observations  of 
morphological  changes.  Conversely,  PET  can  provide  functional  information  that  is  not  available  in  normal 
anatomical  MR  images.  The  goal  of  this  project  is  to  apply  PET  imaging  to  study  the  rapid  biochemical  and 
physiological  response  of  tumors  to  PDT  and  to  use  MR  to  monitor  the  anatomical  and  morphological  changes  of 
tumors. 

The  combination  of  pPET  and  high-resolution  MRI  has  several  advantages.  First,  MRI  scans  provide  anatomical 
reference  to  the  PET  images.  Second,  registration  of  PET  and  MRI  images  can  enhance  our  ability  to  visualize  the 
location  of  the  FDG  uptake.  Third,  MRI  provides  tumor  shape  and  size  information  that  can  be  used  to  improve  the 
accuracy  of  the  PET  data  analysis  such  as  drawing  region  of  interests  (ROIs)  and  performing  quantitative  analyses. 
Fourth,  MRI  can  be  used  to  correct  PET  data  for  partial  volume  effects  to  clarify  that  the  PET-measured  changes 
induced  by  PDT  are  due  to  metabolic  and  hemodynamic  changes  and  not  artifacts  due  to  changes  in  tumor  size. 

In  this  study,  we  focus  on  image  registration  methods  for  the  alignment  of  MRI  and  pPET  images.  Several  reports 
described  the  registration  of  MRI  and  PET  for  the  cat  brain,8  the  rat  brain,9  a  brain  phantom,10  and  radiotherapy 
planning.11  There  is  no  report  on  mutual  information  image  registration  of  jiPET  and  MRI  for  the  study  of 
photodynamic  therapy.  We  acquired  both  PET  and  MR  images  from  mice  with  tumors  and  performed  over  40 
registration  experiments.  Evaluation  results  from  visual  inspection  and  quantitative  measurements  are  reported. 


2.  REGISTRATION  ALGORITHM 


2.1  Similarity  Measurements 


Based  on  our  previous  experience,12’13  we  chose  normalized  mutual  information  (NMI)  as  the  similarity  measure  in 
our  registration  because  it  is  robust  and  suitable  for  multi-modality  image  registration.14,15  One  image  R  is  the 
reference ,  and  the  other  F  is  floating .  Their  normalized  mutual  information  (NMI)  is  given  by  the  following 
equation.15 


NMI(RfF) 


2MI(R,F) 
H(R)  +  H(F) 


where 


r 

H(F)=-Jj>Fif)\oWF{f) 

f 


MI{R,F)  =  ^  pRF  (r ,  /)  log 

rj 


pR(r)pF{f) 


The  joint  probability  pRF(r,f )  and  the  marginal  probabilities  pR(r )  of  the  reference  image  and  pF(f )  of  the 
floating  image,  can  be  estimated  from  the  normalized  joint  intensity  histograms.  When  two  images  are  geometrically 
aligned,  NMI  is  maximal.15 


2.2  Registration  Algorithm 

The  input  data  include  the  PET  transmission,  emission,  and  high-resolution  MR  images.  We  normally  discretize  the 
image  intensity  to  256  levels.  We  combine  the  PET  transmission  and  emission  images  and  form  one  data  set  by  taking 
a  weighted  sum.  We  assume  that  there  is  no  movement  between  the  transmission  and  emission  scans.  We  use  the 
combined  PET  data  and  the  high-resolution  MR  image  for  mutual  information  registration. 

We  used  rigid  body  transformation  (three  translations  and  three  rotations)  and  trilinear  interpolation  as  described 
previously.16  For  optimization,  we  used  the  downhill  simplex  method  of  Nelder  and  Mead.17  Optimization  of 
similarity  ends  either  when  the  maximum  number  (800)  of  calculations  is  reached  or  the  fractional  change  in  similarity 
function  is  smaller  than  a  tolerance  (0.001).  Typically  the  latter  is  achieved  within  about  200  iterations.  Our  very  first 
initial  guess  is  all  zeros  for  the  3  displacements  and  3  angles. 


3.  EXPERIMENTAL  METHODS 


3.1  Animal  Preparation 

RIF  (Radiation-induced  fibrosarcoma)- 1  cells  were  grown  as  monolayers  in  E-MEM  supplemented  with  15%  fetal 
bovine  serum  (ref.  18).  Prior  to  inoculation,  C3H/HeN  mice  were  shaved  and  depilated.  Two  tumors  were  initiated  in 


each  mouse  by  injection  of  105  -  106  RIF-1  cells  intradermally  on  the  shoulder  flanks,  as  far  from  the  bladder  and 
kidneys  as  possible  to  minimize  spillover  in  PET  images.  Tumors  were  treated  and  imaged  when  they  reach  3-5  mm 
in  diameter,  which  required  7-10  days  after  implantation.  Animals  were  given  the  photosensitizer  Pc  4  (1  mg/kg)  by 
tail  vein  injection.  After  24  hours,  one  of  the  tumors  was  exposed  to  red  light  (670  nm)  from  a  diode  laser  (150  J/cm2; 
150  mW/cm2). 

Three  animals  were  studied  for  pPET  and  MR  imaging.  We  know  from  experience  that  neither  the  light  nor  the 
photosensitizer  alone  produces  any  response,  so  initially,  one  of  the  two  tumors  in  each  animal  served  as  a  control 
(receiving  photosensitizer  but  no  light),  and  the  other  tumor  was  exposed  to  laser  light. 

3.2  MR  Image  Acquisition 

Two  days  after  photosensitizer  injection,  the  animals  were  taken  to  the  MR  imaging  facility.  The  mouse  images  were 
acquired  using  a  Siemens  Sonata  1 .5  T  scanner  (Siemens  Medical  Systems,  Erlangen,  Germany).  A  custom-designed 
whole-body  mouse  coil  (2-element  phased-array,  ID  =32  mm)  was  used  to  minimize  noise  levels.  A  Tl-weighted 
spin  echo  pulse  sequence  (TR/TE=600/13ms)  with  a  slice  thickness  of  1-mm  was  used  to  generate  high-resolution 
coronal  images  (Matrix  =  256  x  120,  FOV  =  80  x  36-mm).  The  number  of  signal  averages  was  typically  set  at  six  to 
obtain  low  noise  images.  In  these  T1  images,  the  tumors  are  clearly  delineated  by  the  bright  subcutaneous  fat  signal. 
During  each  imaging  session,  the  animals  were  mounted  on  a  plastic  holder  and  were  provided  with  a  continuous 
supply  of  2%  isoflurane  (EZAnesthesia,  Palmer,  PA)  in  oxygen  to  minimize  motion  artifacts  in  MR  images. 

3.3  Micro-PET  Image  Acquisition 

After  MR  image  acquisition,  the  animals  with  the  plastic  holder  and  the  lasers  were  taken  to  the  PET  imaging  facility. 
We  used  a  MicroPET  R4  scanner  (Concorde  Microsystems,  Inc,,  Knoxville,  TN  37932)  designed  specifically  for 
imaging  small  rodents.20  The  in  vivo  functional  imaging  with  microPET®  allows  both  serial  and  longitudinal  studies 
to  be  conducted  in  the  same  animal.  We  followed  a  single  animal  over  90-minute  period  of  time  and  monitored  the 
response  of  PDT  and  the  outcome. 

We  used  ,8F-FDG  (fludeoxyglucose)  that  is  the  standard  radiopharmaceutical  used  in  PET  scanning  for  tumor 
diagnosis  and  assessment,  for  cerebral  glucose  metabolism,  and  for  myocardial  metabolic  assessment.  It  has  become  a 
standard  commodity  and  is  obtained  commercially  as  well  as  being  produced  on  the  premises.  It  was  produced  for  this 
experiment  in  standard  fashion.21,22 

The  control  tumor  was  shielded  with  black  cloth,  and  the  animals  were  placed  in  plastic  holders  to  restrain  them 
during  imaging.  The  mouse  was  studied  for  FDG  accumulation,  blood  flow  and  blood  volume.  Depending  upon  the 
simulations,  l8F-FDG  was  injected  as  a  bolus  or  by  continuous  infusion  into  the  tail  vein.  About  6  min  later,  laser  light 
was  focused  onto  a  1-cm  spot  encompassing  the  non-shielded  tumor.  FDG  accumulation  in  both  tumors  was  measured 
during  the  15-min  light  exposure. 

We  acquired  both  transmission  and  emission  images  from  the  same  mouse.  Since  the  animal  did  not  move  during 
the  image  session,  we  assume  the  there  is  no  movement  between  the  two  scans.  The  PET  images  include  63  transverse 
slices  covering  the  whole  mouse.  Each  slice  has  128xl28-pixel  with  an  in-plane  pixel  size  of  0.85  x  0.85-mm  and  a 
thickness  of  1.2  mm. 


3.4  Preprocessing 


The  input  MR  volume  is  a  2D  MR  acquisition  giving  256  xl20  with  an  in-plane  resolution  of  0.3x0.3-mm  and  a  slice 
thickness  of  1.0-mm.  Twenty  nine  coronal  slices  over  a  field  of  view  cover  the  whole  mouse.  Using  tri-linear 
interpolation,  we  create  isotropic  voxels  of  0.3  mm  on  a  side  for  both  PET  and  MR  image  volumes.  We  optionally 
discretize  the  intensity  to  256  levels.  We  use  IDL  (Interactive  Data  Language,  Research  System  Inc.,  Boulder,  CO)  as 
the  programming  language. 

For  purposes  of  registration,  we  optionally  cropped  image  slices  that  were  not  of  interest.  For  example,  the  tumors 
were  on  the  mouse  back  near  the  shoulder,  we  cropped  out  images  at  the  abdomen  and  tail.  Before  cropping,  the  image 
volume  was  350x250x250-voxel  covering  the  whole  mouse.  After  cropping,  we  created  a  volume  with  128x128x248- 
voxel  near  the  region  of  interest.  Cropping  processing  can  bring  two  advantages  for  the  mouse  registration.  First, 
cropping  out  regions  that  are  not  of  interest  can  increase  image  consistency  for  the  mutual  information  registration. 
Since  the  mouse  body  is  very  flexible,  the  deformation  at  the  abdomen  can  cause  inconsistency  for  the  rigid  body 
registration.  Second,  the  small  number  of  voxels  after  cropping  can  increase  the  speed  of  image  registration. 


Figure  1.  Visual  inspection  of  the  kidney  registration.  Image  on  the  left  is  the  high-resolution  MR  image.  The 
white  arrows  indicate  the  left  and  right  kidneys  of  the  mouse.  Image  at  the  center  is  the  PET  FDG  image.The 
white  arrows  indicate  the  kidneys.  Image  on  the  right  is  the  overlay  of  the  two  images,  which  indicate  good 
registration  at  the  organs.  Images  are  from  the  Mouse  No. 3. 


Figure  2.  Registration  of  tumors.  Image  on  the  left  is  the  coronal  MR  image.  The  white  arrows  indicate  two 
tumors  on  the  back  of  the  mouse.  Image  at  the  center  is  the  PET  FDG  image.  The  white  arrows  indicate  two 
tumors.  Image  on  the  right  is  the  overlay  image.  The  tumors  are  registered  well.  Images  are  from  the  Mouse 


No.3. 


Bins  for  both  MR  and  PET  Transmission  Images 


Figure  3.  The  effect  of  intensity  scaling  on  image  registration.  The  X-axis  is  the  gray  levels  from  32  to  512. 
The  Y-axis  it  the  normalized  mutual  information  values  between  MR  and  PET  transmission  images.  The  dash 
line  indicates  the  NMI  values  before  registration.  The  real  line  is  the  NMI  values  after  registration. 


4.  Results  and  Discussion 


4.1  Visual  Inspection 

Figure  1  shows  the  registration  of  the  left  and  right  kidneys  of  a  mouse.  The  MR  image  provides  the  shape  of  the 
kidneys.  The  PET  image  shows  the  FDG  uptake  of  the  organs.  The  fused  image  combines  both  anatomic  and 
functional  information.  We  also  examined  other  slices  in  different  positions.  The  kidneys  were  well-aligned  in  three 
dimensions.  The  kidneys  are  good  anatomic  markers  for  registration  evaluation.  Other  feature  organs  such  as  the 
bladder  and  heart  were  also  aligned. 

Figure  2  shows  the  registration  of  two  tumors.  The  high  resolution  MR  image  contains  the  anatomic  structure  of 
the  tumors.  The  PET  image  shows  the  radioactivity  of  the  tumors.  The  overlay  image  indicates  good  registration  of  the 
tumors.  We  also  used  other  visualization  methods  found  in  MIM  ™  (MIMvista  Corp.,  Cleveland,  Ohio  44122)  for  the 
image  display  and  analysis. 

4.2  Effects  of  Intensity  Scaling 

We  tested  the  effect  of  intensity  scaling  on  image  registration.  Since  PET  images  are  stored  as  float  data,  we  normally 
discretize  the  data  to  256  gray  levels  for  image  display  and  processing.  We  then  use  the  scaled  data  and  apply  the 
mutual  information  algorithm.  Different  intensity  scaling  creates  different  joint  histograms  that  affect  the  mutual 
information  calculation.  We  performed  registration  experiments  using  different  intensity  scaling  such  as  512,  256, 
128,  64,  or  32  bins  for  both  volume  data  sets.  Scaling  was  linear  between  zero  and  the  maximum  value.  Registration 
was  examined  by  analyzing  the  normalized  mutual  information  values  and  visual  inspection. 

In  Figure  3,  we  show  the  NMI  values  between  the  PET  transmission  image  and  MR  image  volume.  First,  the  NMI 
values  increased  after  registration.  Second,  when  the  bin  size  is  256,  the  NMI  is  the  maxim  indicating  the  gray  level  of 
256  is  the  best  for  the  registration.  The  visual  inspection  confirmed  the  results. 

4.3  Quantitative  Evaluation 

We  used  features  identifiable  in  both  PET  and  MR  images  to  evaluate  the  mouse  registration.  We  manually  segmented 
feature  organs  such  as  the  left  and  right  kidneys  and  the  bladder  in  each  slice  for  both  MR  and  PET  image  volumes. 
We  calculated  the  area  and  the  center  point  for  each  slice.  Then,  we  compute  the  three-dimensional  centroid  of  the 
organs.  To  evaluate  the  registration  error,  we  computed  the  distance  between  corresponding  centroids.  For  three  mice, 
the  registration  error  is  1.5  ±  0.4  mm,  about  2  pPET  pixels.  More  importantly,  pPET  resolution  is  about  2  nnn.20 

4.4  Combining  PET  Emission  and  Transmission  Images 

We  found  that  only  using  PET  emission  image  led  to  the  registration  failure  and  that  it  is  important  to  add 
transmission  image  for  the  registration.  We  think  the  anatomic  information  from  transmission  image  may  be  good  for 
the  mutual  information  image  registration.  However,  increasing  the  percentage  of  transmission  image  from  10%  to 
100%  made  no  significant  difference. 


5.  CONCLUSION 


We  created  a  rigid  body  image  registration  method  for  jiPET  and  high  MR  images  for  a  new  application  in  small 
animal  imaging.  The  registration  and  fusion  provided  both  functional  and  anatomic  information  that  could  be  useful 
for  the  potential  application  in  photodynamic  therapy.  We  are  going  to  perform  more  imaging  experiments  and  we  are 
developing  deformable  registration  method  for  small  animals. 
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Abstract  —  We  created  an  automatic  three-dimensional 
registration  algorithm  for  magnetic  resonance  images  of 
carotid  vessels.  Potential  applications  include  atherosclerotic 
plaque  characterization  and  plaque  burden  quantification 
vector-based  segmentation  using  dark  blood  MR  images  having 
multiple  contrast  weightings  (proton  density  (PD),  Tl,  and  T2). 
Another  application  is  measurement  of  disease  progression  and 
regression  with  drug  trials.  We  used  mutual  information 
registration  algorithm  to  compensate  movements  between 
image  acquisitions.  PD,  Tl,  and  T2  images  were  acquired  from 
patients  and  volunteers  and  then  matched  for  image  analysis. 
Visualization  methods  such  as  contour  overlap  showed  that 
vessels  well  aligned  after  registration.  Distance  measurements 
from  the  landmarks  indicated  that  the  registration  method 
worked  well  with  an  error  of  1.09  ±  0.42  mm. 

Keywords — Image  registration,  dark  blood  MR  image, 
carotid  plaque  classification 


I.  Introduction 

Atherosclerotic  disease  of  the  carotid  artery  is  the 
leading  cause  of  stroke  [1].  MRI  has  emerged  as  a  potential 
leading  in  vivo  imaging  modality  for  atherosclerotic  plaque 
characterization  [2]  [3],  Black  blood  high-resolution  MRI 
techniques  with  multiple  contrast  weightings  (proton  density 
(PD),  Tl,  and  T2)  have  been  shown  to  be  useful  for 
atherosclerotic  plaque  characterization  and  plaque  burden 
assessment  [4]. 

Quantification  during  disease  progression  and  after 
therapeutic  intervention  may  improve  our  knowledge  of  the 
natural  history  of  the  disease  and  lead  to  improved 
therapeutic  strategies.  However,  the  spatial  location 
mismatching  of  different  scans  from  serial  examination  will 
impair  the  accuracy  of  quantified  analysis  of  atherosclerotic 
disease  using  MRI.  Image  registration  has  the  potential  to 
improve  the  quantification  and  characterization  [5]. 
Combination  of  multiple  images  such  as  proton  density 
(PD),  Tl,  and  T2  could  verify  lesions  and  provide  more 
information  for  diagnosis  and  treatment.  Further, 
registration  of  serial  examinations  can  follow  up  the 
progression  and  regression  of  diseases  [6].  In  this  study,  we 
perform  registration  experiments  for  MR  images  from 
patients  with  carotid  stenosis  and  volunteers. 


II.  Methodology 
A .  Registration  A  Igorithm 


We  are  investigating  voxel-based  methods  for  automatic 
three-dimensional  (3D)  registration  because  they  do  not 
depend  on  possibly  inaccurate  image  segmentation  and 
because  they  can  be  extended  to  non-rigid  registration  [7]- 
[9]. 

We  used  normalized  mutual  information  (NMI)  as  the 
similarity  measures  in  our  registration  because  they  are 
robust  and  suitable  for  multi-modality  image  registration 
[7].  Suppose  one  image  R  is  the  reference ,  and  the  other  F  is 
floating .  NMI  is  given  by  the  following  equation  [7]. 


NMI(R,F) 


2  MI(R,F) 
H(R)  +  H(F) 


where 


H{R)=-^lpR{r)\o%pR(r)  ’  H(F)=-'£pF(f)\ogpF(f) 
r  f 


MI(R,F )  =  ^pRF(r,f)  log . 

PR(r)‘  PF(f) 


rj 


The  joint  probability  pRfr,f) and  the  marginal  probabilities 
pR(r)  of  the  reference  image  and  pF(f)  °f  the  floating  image, 


can  be  estimated  from  the  normalized  joint  intensity 
histograms.  When  two  images  are  geometrically  aligned, 
NMI  is  maximal. 

We  used  rigid  body  transformation  (three  translations 
and  three  angles)  and  trilinear  interpolation.  For 
optimization,  we  use  the  downhill  simplex  method  of  Nelder 
and  Mead  [10].  Optimization  of  alignment  ends  either  when 
the  maximum  number  of  NMI  calculations  is  reached 
(typically  500)  or  the  fractional  change  in  NMI  is  smaller 
than  a  tolerance  (typically  0.0001).  Our  very  first  initial 
guesses  are  all  zeros  for  the  3  displacements  and  3  angles. 


B.  Im age  A  cq nisi tio ns 


All  MR  scans  were  conducted  on  a  1.5  T  system 
(Magnetom  Sonata,  Siemens  Medical  Solutions,  Erlangen, 
Germany)  with  a  custom-built  phased  array  coil  to  improve 
the  local  image  signal -to-noise  ratio.  Patients  were 
positioned  supine  on  the  scanner  table.  After  axial,  sagittal 
and  coronal  localizer  images  were  acquired;  a  set  of  double 
oblique  localizer  images  was  then  acquired  to  monitor  the 
phased  array  coil  position  and  to  roughly  identify  the  carotid 
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III.  Results 


artery  bifurcation.  A  transverse  three-dimensional  (3D) 
multiple  overlapping  thin  slab  angiography  (MOTSA) 
sequence  with  TR/TE/fl ip/partition  thickness  of  20ms/ 
3.4ms/25°/lmm,  was  used  to  locate  the  exact  level  of  the 
carotid  bifurcation.  Dark  blood  images  were  then  obtained 
using  ECG-triggered  double  inversion  recovery  (DIR)  turbo 
spin  echo  sequences.  The  imaging  parameters 
(TR/TE/TI/NSA/thickness/FOV)  were  as  follows:  1R- 
R/7.1ms/500ms/2/3mm/13cm  (Tl),  2R-R/7.1ms/600ms/2/ 
3mm/13cm(PD),  2R-R/68ms/600ms/2/3mm/13cm  (T2).  Fat 
saturation  was  applied  for  all  dark  blood  images.  The  in 
plane  resolution  was  0.51x  0.51mm2.  We  acquired  PD,  Tl, 
and  T2  images  from  one  patient  PI  and  two  volunteers  SI 
and  S2. 

C.  Registration  Experiments 

We  have  nine  pairs  of  volumes  for  registration 
experiments.  For  each  subject,  we  have  PD,  Tl  and  T2 
image  volumes.  We  used  PD  as  the  reference  image  and  we 
registered  Tl  and  T2  with  PD.  We  also  matched  T2  with  Tl 
to  test  the  registration  algorithm.  There  are  three  registration 
experiments  for  each  subject.  We  performed  two  registration 
trials  for  volunteers  SI  and  S2.  In  total,  we  conducted  15 
registration  experiments,  three  for  the  patient  PI  and  three 
for  each  of  the  volunteers  SI  and  S2. 

D.  Registration  Evaluation 

We  used  visual  inspections  to  evaluate  the  registration. 
We  used  RegViz,  a  program  written  in  IDL  (Interactive  Data 
Language,  Research  System  Inc.,  Boulder,  CO)  and  created 
in  our  laboratory  for  visualizing  and  analyzing  registered 
image  volumes.  First,  we  manually  segmented  carotid  vessel 
walls  in  image  slices  and  copied  them  to  corresponding 
slices.  This  enabled  visual  determination  of  the  overlap  of 
vessel  walls  over  the  entire  volume.  Second,  color  overlay 
displays  were  used  to  evaluate  overlap  of  structures.  One 
image  was  rendered  in  gray  and  the  other  in  the  “hot-iron” 
color  scheme  available  in  IDL.  To  visualize  potential 
differences,  it  was  quite  useful  to  interactively  change  the 
contribution  of  each  image  using  the  transparency  scale. 
Third,  we  used  a  sector  display,  which  divided  the  reference 
and  registered  images  into  rectangular  sectors  and  created  an 
output  image  by  alternating  sectors  from  the  two  input 
images.  Even  subtle  shifts  of  edges  would  be  clearly  seen. 

We  evaluated  registration  of  the  vessels  by  measuring 
the  displacement  of  the  landmarks  following  registration. 
We  used  the  left  and  right  bifurcations  as  the  landmarks.  To 
measure  displacements,  we  navigated  transverse,  coronal, 
and  sagittal  MR  images  slice-by-slice  to  search  bifurcations 
in  both  reference  and  registered  volumes.  Using  consistent 
rules  and  magnified  images,  a  radiologist  used  a  cursor  to 
identify  unique  features  on  both  images.  The  3D  locations 
for  each  landmark  were  recorded  and  3D  distance  between 
corresponding  landmarks  is  computed. 


A.  Visual  Inspection 

We  determined  the  quality  of  vessel  registration  by 
visually  examining  all  image  slices  of  registered  volume 
pairs  using  one  or  more  of  the  methods  found  in  RegViz.  A 
typical  example  is  shown  in  Fig.l  where  the  contour  overlap 
is  excellent  and  probably  within  the  manual  segmentation 
error.  Other  transverse  images  were  also  well  aligned 
indicating  that  the  registration  was  successful  in  three 
dimensions.  Other  visual  inspection  techniques  such  as  color 
overlay  and  sector  display  also  demonstrate  excellent 
registration. 

B.  Assessments  of  Bifurcations 

Following  registration,  we  measured  3D  distances 
between  corresponding  bifurcations.  Registration  results  are 
shown  in  Table  I.  The  average  error  across  the  three  subjects 
with  15  registration  experiments  is  only  1.09  ±0.42  mm. 
The  isotropic  voxel  size  of  the  volumes  is  0.51  mm.  The 
measured  error  reported  for  NMI  registration  is  probably 
overestimated  due  to  landmark  location  error. 


TABLE  I 

DISPLACEMENT  OF  BIFURCATIONS 
MEAN  (STANDARD  DEVIATION)  IN  mm 


PD-T1 

PD-T2 

T1-T2 

Left  bifurcation 

1.04  (0.63) 

1.49  (0.56) 

1.02  (0.27) 

Right  bifurcation 

0.97  (0.34) 

1.04  (0.24) 

0.98(0.31) 

C.  Implementation  Issues 

The  algorithm  was  quite  robust  for  nine  volume  pairs  in 
this  study.  Each  call  to  the  Simplex  optimization  resulted  in 
120  to  250  NMI  evaluations  before  the  tolerance  (0.0001) 
was  reached.  The  time  for  a  single  registration,  typically 
5  minutes  for  the  volumes  with  256x256x59-voxels  on  a 
Pentium  IV,  1.8GHz  CPU,  with  1  Gbytes  of  memory,  could 
probably  be  greatly  improved  with  optimized  C  code  rather 
than  IDL. 


IV.  Discussion  &  Conclusion 

We  have  developed  an  automatic  volume  registration 
algorithm  for  multiple  contrast  weighted  MR  images  of 
carotid  vessels.  Our  internal  measures  showed  the 
registration  is  quite  robust  and  accurate.  It  will  probably  be  a 
useful  tool  for  many  applications  of  interest  in  vascular 
imaging. 
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Fig.l.  MR  images  of  carotid  vessels.  The  top  images  from  left  to  right  are  proton  density  (PD),  Tl,  and  T2  images,  respectively.  Both  T1  and  T2  images 
were  registered  with  the  PD  image.  The  rectangular  regions  are  magnified  and  displayed  at  the  bottom.  The  carotid  vessel  walls  were  manually  segmented 
from  the  PD  image  (left)  and  copied  to  the  Tl  (center)  and  T2  images  (right).  The  contour  overlaps  at  the  bottom  shows  that  the  vessel  boundaries  from  PD 
well  aligned  with  those  in  Tl  and  T2  images.  Images  are  from  Volunteer  SI . 
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Abstract  —  Small  animal  imaging  can  provide  high- 
throughput  phenotypic  data  for  functional  genomic  studies  and 
better  mechanistic  understanding  of  disease.  Fusion  of 
anatomical  and  functional  images  will  aid  interpretation  of 
functional  images  having  relatively  little  anatomical  detail.  In 
this  study,  we  are  investigating  automatic  registration  and 
fusion  visualization  methods  for  micro-CT  and  SPECT  images 
of  rat  lung.  The  immediate  application  is  studies  of  pulmonary 
perfusion  in  a  healthy  rat  and  one  with  an  occluded  left 
pulmonary  artery.  Registration  experiments  were  performed 
on  images  acquired  from  rats  and  a  phantom.  Fusion 
visualization  showed  excellent  registration  results.  Quantitative 
measures  such  as  distances  and  perimeters  from  phantom 
results  show  that  the  registration  is  quite  accurate. 

Keywords — Image  registration  and  fusion,  small  animal 
imaging,  SPECT,  micro-CT,  lung  perfusion 


I.  INTRODUCTION 

Small  animal  imaging  is  a  fast  growing  field  that  has 
numerous  applications  in  the  studies  of  functional  genomics, 
the  biology  of  disease,  and  therapeutics  [1][2].  Since 
commonly,  functional  imaging  modalities  such  as  single 
photon  emission  tomography  (SPECT),  positron  emission 
tomography  (PET)  and  functional  magnetic  resonance 
imaging  (fMRI)  have  little  anatomic  information,  images 
acquired  from  computed  tomography  (CT)  or  MRI  are  used 
to  provide  structural  identification  and  localization  of 
organs/regions  of  interest  and  may  also  provide  additional 
diagnostic  information  [5],  In  this  particular  study,  we 
investigate  automatic  image  registration  and  fusion 
visualization  methods  applied  to  SPECT  and  micro-CT 
images  for  this  new  important  application.  The  model 
application  used  to  demonstrate  the  registration/fusion 
methods  is  the  localization  of  a  lung  perfusion  image 
obtained  with  SPECT  within  the  anatomical  lung  field 
image  obtained  with  micro-CT.  We  perfonned  registration 
experiments  using  image  volumes  acquired  both  from  rats 
and  a  gamma  emitting,  x-ray  absorbing  phantom. 


II.  MATERIALS  AND  METHODS 

A.  Registration  Algorithm 

Based  on  our  previous  experiences  [6] [7],  we  chose 
normalized  mutual  information  (NMI)  as  the  similarity 
measures  in  our  registration  because  it  is  robust  and  suitable 
for  multi-modality  image  registration  [8],  We  used  rigid 
body  transformation  (three  translations  and  three  angles)  and 
trilinear  interpolation.  For  optimization,  we  used  the 
downhill  simplex  method  of  Nelder  and  Mead  [9], 
Optimization  of  alignment  ends  either  when  the  maximum 
number  of  NMI  calculations  is  reached  (typically  500)  or  the 
fractional  change  in  NMI  is  smaller  than  a  tolerance 
(typically  0.0001).  Our  very  first  initial  guesses  are  all  zeros 
for  the  three  displacements  and  three  angles.  We  use  IDL 
(Interactive  Data  Language,  Research  System  Inc.,  Boulder, 
CO)  as  the  program  language. 

B .  Image  Acquisitions 

We  used  the  SPECT  and  micro-CT  systems  developed 
by  Marquette  University  and  Medical  College  of  Wisconsin 
as  described  previously  [4].  For  SPECT  data  acquisition,  a 
mobile  gamma  camera  was  positioned  in  front  of  the 
specimen  stage  and  perpendicular  to  the  x-ray  beam.  The 
center  of  the  pinhole  collimator  was  positioned  at  the  height 
of  the  x-ray  source  focal  spot.  The  SPECT  data  were 
acquired  in  a  step-and-shoot  fashion  using  128  equiangular 
increments  over  a  full  360°  at  40  sec/view  and  reconstructed 
using  ordered  subset  expectation  maximization  (OSEM)  [4]. 
Subsequently,  micro-CT  images  were  acquired  at  1° 
increments  over  360°.  Seven  frames  were  averaged  at  each 
position.  This  data  was  reconstructed  using  Feldkamp  [3] 
conebeam  reconstruction. 

We  performed  imaging  experiments  with  a  phantom 
consisting  of  three  test  tubes  filled  with  technetium 
(Tc99m),  which  were  embedded  within  a  larger  cylinder. 
The  phantom  was  placed  on  the  specimen  stage  and  imaged 
as  described  above.  After  the  SPECT  acquisitions  were 
completed,  the  phantom  was  imaged  using  micro-CT. 

We  acquired  image  sets  from  two  rats  that  were 
anesthetized  with  40mg  per  kg  pentobarbital  sodium  [4]. 
One  rat  underwent  surgery  in  which  the  left  pulmonary 
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artery  was  occluded  8  days  prior  to  the  imaging.  The  other 
rat  was  untreated.  A  femoral  venous  catheter  was  used  to 
inject  0.6  ml  of  Tc99m  labeled  macro-aggregated  albumin 
with  a  total  radioactivity  of  2.0  mCi.  After  allowing  the 
compound  enough  time  to  distribute  and  accumulate  in  the 
rat’s  lungs,  the  animals  were  sacrificed  using  an  overdose  of 
the  anesthetic.  The  rats  were  then  placed  head  down  in  a 
plastic  tube  with  diameter  of  52mm,  and  were  positioned  on 
the  specimen  stage.  The  center  of  rotation  was  set  so  that  the 
right  and  left  lungs  were  within  the  field  of  view  at  all 
angular  positions.  SPECT  and  micro-CT  data  were  acquired 
as  described  above. 

C.  Registration  Experiments 

Before  we  performed  image  registration  experiments, 
we  preprocessed  both  micro-CT  and  SPECT  image 
volumes.  The  original  CT  image  volume  has  512x512x497 
voxels  with  497  transverse  slices.  The  voxel  size  is 
0.1 0x0.1 0x0. 10-mm.  The  SPECT  image  volume  is 
128x128x1 28-voxel  with  voxel  size  of  0.48x0.48x0.48-mm. 
We  resample  CT  images  and  interpolated  SPECT  images 
generating  two  volumes  with  the  same  voxel  size  of 
0.2x0. 2x0. 2-mm.  We  cropped  both  SPECT  and  CT  images 
that  have  no  signal  of  interest.  The  final  volume  images  are 
256x256x156  for  phantom  data  and  256x256x232  for  rats, 
respectively.  We  transform  CT  volumes  and  registered  with 
corresponding  SPECT  volumes  using  our  3D  registration 
algorithm. 

D.  Registration  Evaluation 

We  used  visual  inspections  to  evaluate  the  registration. 
We  used  RegViz,  a  program  written  in  IDL  and  created  in 
our  laboratory  for  visualizing  and  analyzing  registered 
image  volumes.  First,  color  overlay  displays  were  used  to 
evaluate  overlap  of  structures.  One  image  was  rendered  in 
gray  and  the  other  in  the  “hot-iron”  color  scheme  available 
in  IDL.  To  visualize  potential  differences,  it  was  quite  useful 
to  interactively  change  the  contribution  of  each  image  using 
the  transparency  scale.  Second,  we  manually  segmented 
regions  of  interest  (ROI)  in  image  slices  and  copied  them  to 
corresponding  slices.  This  enabled  visual  determination  of 
the  overlap  of  ROI  over  the  entire  volume.  To  evaluate 
registration  of  the  phantom,  we  manually  segmented  the 
three  tubes  within  the  phantom  from  both  CT  and  SPECT 
images.  We  calculated  the  central  positions  and  computed 
the  distance  between  corresponding  central  positions  and  the 
perimeters  of  the  segmented  tubes. 


III.  Results 

The  fusion  visualization  of  registered  micro-CT  and 
SPECT  images  shows  that  the  tubes  within  the  phantom  are 
well  matched  (Fig.lc).  The  distances  between  corresponding 
central  points  are  less  than  0.25-mm  and  the  perimeter 


differences  are  less  than  0.33-mm.  Hence,  the  phantom 
registration  is  quite  accurate. 

We  determined  the  registration  quality  of  the  rat  images 
by  visually  examining  all  image  slices  of  registered  volume 
pairs  using  one  or  more  of  the  methods  found  in  RegViz.  A 
typical  example  is  shown  in  Fig.l  where  the  color  overlays 
show  that  the  rat  lung  aligned  very  well.  Other  transverse 
images  were  also  well  aligned  indicating  that  the  registration 
was  successful  in  three  dimensions.  Contour  overlap  and 
sector  display  also  demonstrate  excellent  registration. 


IV.  Discussion  &  Conclusion 

Using  our  three  dimensional  automatic  registration 
method,  we  successfully  registered  high  resolution  micro- 
CT  images  with  lung  perfusion  SPECT  images  of  the  rat. 
Phantom  results  show  that  the  registration  is  quite  accurate. 
As  we  are  performing  more  rat  imaging  experiments,  we 
believe  that  the  registration  and  fusion  visualization  method 
could  be  a  useful  tool  for  many  applications  in  small  animal 
imaging. 
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Fig.l.  Images  of  Micro-CT  (a,d,g),  SPECT  (b,e,h),  and  fusion  visualization  (c,f,i).  a,b,c:  Images  of  the  phantom.  In  the  fused  image  (c),  the  three  tubes 
within  the  phantom  are  well  aligned.  d,e,f:  Normal  lung  images  showing  the  lung  field  (d)  and  perfusion  (e)  of  both  the  left  and  right  lungs  of  the  rat.  g,h,i : 
Images  obtained  from  a  rat  with  occluded  left  pulmonary  artery.  Perfusion  (h)  is  observable  only  in  the  right  lung  in  this  case.  The  registered  fusion 
visualization  (i)  excellently  represents  both  structural  and  functional  information. 
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ABSTRACT 

We  are  investigating  interventional  MR]  (iMRI)  guided  radiofrequency  thermal  ablation  for  the  minimally  invasive 
treatment  of  prostate  cancer.  Nuclear  medicine  and  MR  spectroscopy  can  detect  and  localize  tumor  in  the  prostate  not 
reliably  seen  in  MR.  We  are  investigating  methods  to  combine  the  advantages  of  functional  images  such  as  SPECT  with 
iMRI-guided  treatments.  Our  concept  is  to  first  register  the  low-resolution  functional  images  with  a  high  resolution  MR 
volume.  Then  by  registering  the  high-resolution  MR  image  with  live-time  iMRI  acquisitions,  we  can,  in  turn,  map  the 
functional  data  and  high-resolution  anatomic  information  to  iMRI  images  for  improved  tumor  targeting.  In  this  study, 
we  registered  noisy,  thick  iMRI  image  slices  with  high-resolution  MR  volumes  and  called  this  slice-to-volume 
registration.  We  investigated  two  similarity  measures,  i.e.,  mutual  information  and  correlation  coefficient,  and  three 
interpolation  methods,  i.e.,  tri-linear,  re-normalized  sine,  and  nearest  neighbor.  To  assess  the  quality  of  registration,  we 
calculated  3D  displacement  on  a  voxel-by-voxel  basis  over  a  volume  of  interest  (VOI)  between  slice-to-volume 
registration  and  volume-to- volume  registration  that  was  previously  shown  to  be  quite  accurate  for  these  image  pairs. 
Over  300  registration  experiments  showed  that  transverse  slice  images  covering  the  prostate  work  best  with  a 
registration  error  of  only  0.4  ±  0.2  mm.  Error  was  greater  at  other  slice  orientations  and  positions.  Since  live-time  iMRI 
images  are  used  for  guidance  and  registered  images  are  used  for  adjunctive  information,  the  accuracy  and  robustness  of 
slice-to-volume  registration  is  very  probably  adequate. 

Keywords:  Image  registration,  medical  imaging,  interventional  magnetic  resonance  imaging  (iMRI),  mutual 
information,  thermal  ablation,  prostate  cancer. 


1.  INTRODUCTION 

We  are  investigating  interventional  MRI  (iMRI)  guided  radiofrequency  (RF)  thermal  ablation  for  the  minimally 
invasive  treatment  of  prostate  cancer.  Nuclear  medicine  and  MR  spectroscopy  can  detect  and  localize  tumor  in  the 
prostate  not  reliably  seen  in  MR.  We  are  investigating  methods  to  combine  the  advantages  of  functional  images  such  as 
SPECT  with  iMRI-guided  treatments.  Our  concept  is  to  first  register  the  low-resolution  functional  images  with  a  high 
resolution  MRI.  Then  by  registering  the  high-resolution  MR  volume  with  live-time  iMRI  acquisitions,  we  can,  in  turn, 
map  the  functional  data  and  high-resolution  anatomic  information  to  iMRI  images  to  aid  tumor  targeting. 

Previous  success  with  registering  one  MR  prostate  volume  to  another  1  encourages  us  to  pursue  this  plan.  We  call 
this  volume-to-volume  registration,  or  W.  We  used  a  rigid  body,  mutual  information  registration  method  with  some 
features  to  improve  robustness.1  We  carefully  evaluated  registration  quality  using  a  variety  of  methods.  For  volume 
pairs  acquired  over  a  short  time  span  from  a  supine  subject  with  legs  flat  on  the  table,  registration  accuracy  of  both 
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prostate  centroids  (typically  <  1  mm)  and  bony  landmarks  (average  1.6  mm)  was  on  the  order  of  a  voxel  ( »  1.4  mm). 
For  volumes  acquired  under  very  different  conditions,  e.g.,  legs  flat  and  legs  raised  into  the  treatment  position,  or  with 
and  without  bladder  or  rectal  filling,  we  obtained  somewhat  larger  prostate  centroids  registration  errors  of  about  3.0 
mm. 

In  this  study,  we  are  investigating  methods  to  register  live-time  iMRI  image  slices  with  a  previously  obtained,  high 
resolution  MRI  volume.  We  call  this  slice-to-volume  (SV)  registration.  Because  of  our  success  with  VV  prostate 
registration,  we  can  determine  SV  accuracy  by  comparing  results  to  VV  registration  for  volume  pairs  having  low  W 
registration  error. 

The  application  of  SV  registration  methods  to  iMRI-guided  treatment  of  prostate  cancer  raises  several  challenges. 
First,  iMRI  images  often  have  lower  signal  to  noise  ratio  (SNR)  than  diagnostic  MR  images  because  of  the  emphasis  on 
fast  imaging  and  because  of  the  typically  lower  field  strength  of  open  iMRI  magnets.  Second,  a  single  slice,  or  a  few 
slices,  provides  many  fewer  structures  than  an  entire  volume  for  voxel  based  matching.  Third,  the  prostate  can  move 
relative  to  the  pelvic  bones  due  to  changes  in  rectal  and  bladder  filling 2,3  or  changes  in  patient  posture  for  treatment.1 
That  is,  alignment  of  the  pelvic  bones,  prominent  anatomical  features  in  MR  gray-scale  images,  does  not  necessarily 
ensure  that  the  prostate  is  aligned.  Fourth,  the  normal  prostate  is  a  small  organ;  when  healthy,  it  measures  only  ~  3.8  cm 
in  its  widest  dimension.4  The  small  prostate  is  located  below  the  much  larger  bladder  that  can  change  its  shape  and  size 
during  imaging.  Finally,  times  for  registration  and  algorithm  robustness  are  of  particular  concern  for  this  application. 

Previously  reported  methods  for  SV  registration  were  mainly  applied  to  the  brain  for  applications  of  functional 
MRI,5,6  postmortem  pathology  studies,7  and  anatomical  modeling.8 

We  previously  reported  an  SV  registration  method  for  the  application  of  iMRI-guided  treatment  of  the  prostate.9'12 
In  this  report,  we  further  investigated  the  algorithms  by  comparing  three  interpolation  methods  and  two  similarity 
measures.  We  registered  noisy,  thick  iMRI  image  slices  with  high-resolution  MR  volumes.  Over  300  registration 
experiments  were  performed  with  9  pairs  of  MR  volume  images  acquired  from  three  health  volunteers. 


2.  REGISTRATION  ALGORITHM 


2.1.  Similarity  Measurements 

We  used  two  similarity  measures,  mutual  information  and  correlation  coefficient  (CC),  in  our  registration.  One  image  R 
is  the  reference ,  and  the  other  F  is  floating.  Their  mutual  information  MI(R,F)  is  given  below.13 

MI(R,F)  =  YpRF(r,f)  log  PmSLiII 

Tf  P*(r)pAf) 


The  joint  probability  pRF (r, /)  and  the  marginal  probabilities  pR(r)  of  the  reference  image  and  pF(f)  of  the 
floating  image,  can  be  estimated  from  the  normalized  joint  intensity  histograms.  When  two  images  are  geometrically 
aligned,  MI  is  maximal.13  The  correlation  coefficient  CC(R,F )  is  given  below.14 

T  ,  £W-)- WW-FW) 

-  R('»‘  X™  -  w>r 

Here  R(r )?  F(f)  denote  the  average  intensities  of  the  reference  and  floating  images  and  the  summation  includes  all 
voxels  within  the  overlap  of  both  images. 
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2.2.  Interpolation  Methods 

We  used  three  interpolation  methods,  i.e.,  nearest  neighbor  (NN),  tri-linear,  and  re-normalized  sine  interpolation.15,16 
The  conventional  sine  interpolation  with  a  cosine  Hamming  window  is  described  as  below.15 

/'(W)=XIX/(*,  Y,  Z)  ■  HS(x,  X,  R )  ■  HS(y,  Y,  R )  •  HS(z,  Z,  R) 

X  Y  Z 

With  HS(a,  A,  R)  =  sm^(a~^)]  •  {  1  +  cos[?r(a  -A)/ R  + 1]}, 

2  7t  (a -A) 

where  X,  Y,  Z ,  are  the  coordinates  of  the  original  data  set  (/ );  x,  y,  z  are  the  coordinates  of  the  re-formatting  data  set 
(/’ );  A  is  a  symbol  representing  X,  Y ,  or  Z,  and  a  represents  x,  y ,  or  z ;  and  R  is  the  kernel  size.  The  Hamming  function 
eliminates  problems  with  oscillatory  effects  at  discontinuities  and  guarantees  that  the  convolution  coefficients  fall  of  to 
zero  at  the  edge  of  the  sine  kernel  (i.e.,  at  faf=R+l,  where  R-5  in  this  study).15 

A  previous  report 15  showed  that  re-normalizing  the  interpolation  kernel  for  a  constant  integral  could  make 
significant  improvement  in  performance  of  the  conventional  sine  interpolation.  In  our  implementation,  we  used  the  re¬ 
normalized  sine  interpolation  method.15  We  replaced  HS  in  the  above  equation  with 

HS  (a,  A,  R)  =  HS(a,A,R)I^HS(a,A,R )  =  HS(a,A,R)/\HS{a)\ . 


2.3.  Comparison  of  Different  Similarity  Measures  and  Interpolation  Methods 

We  compared  two  similarity  measures,  CC  and  MI,  at  different  resolutions  in  order  to  determine  their  suitability  for  SV 
registration.  At  1/4  resolution ,  we  resampled  images  so  as  to  give  1/4  number  of  the  voxels  along  each  linear 
dimension.  At  full  resolution ,  we  used  the  full  number  of  voxels.  We  plot  the  two  similarity  measures  as  a  function  of 
translations.  After  two  typical  high-resolution  MR  volumes  were  registered,1  values  were  plotted  with  the  origin  as  the 
optimal  transformation.  We  calculated  CC  and  MI  values  while  moving  the  simulated  iMRI  image  relative  to  the  high- 
resolution  MR  image  along  coronal  (anterior-posterior)  axis.  When  obtaining  floating  images,  we  used  the  three 
different  interpolation  methods. 

Features  of  MI  and  CC  demonstrate  their  suitability  at  high  and  low  resolutions,  respectively.  At  1/4  resolution,  CC 
curves  are  much  smoother  than  MI,  which  is  noisy  and  contains  many  local  maxima  as  shown  in  Figures  1.  At  full 
resolution,  Figures  2  shows  that  MI  curves  are  much  peaked  than  CC,  but  once  again  there  is  high  frequency  noise  in 
the  MI  curves,  far  from  the  optimum,  that  gives  rise  to  local  maxima  that  must  be  avoided.  From  these  figures,  we  infer 
that  CC  is  better  at  low  resolution  and  MI  is  better  at  full  resolution,  when  one  is  close  to  the  optimum  value. 

We  also  compared  the  effects  interpolation  methods.  At  1/4  resolution  (Figure  1),  the  MI  curves  with  sine  had  more 
local  maximums  than  those  with  tri-linear;  however,  CC  plots  with  sine  were  smoother  than  tri-linear.  This  indicates 
that  CC  and  sine  are  possibly  the  best  choices  for  robust  registration  at  low  resolution.  At  full  resolution  (Figure  2),  MI 
and  sine  gave  the  sharpest  peak  at  the  optimum;  nearest  neighbor  interpolation  gave  a  flat  peak  with  a  width  of  one 
voxel;  and  tri-linear  gave  a  result  between  the  other  two.  These  results  imply  that  MI  and  sine  interpolation  should 
provide  the  best  registration  accuracy  at  full  resolution.  However,  tri-linear  interpolation  is  about  100  times  faster  than 
sine  interpolation  in  our  implementations. 
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Figure  1.  Similarity  functions  are  plotted  as  a  function  of  translations  at  the  1/4  resolution  in  the  multi-resolution  registration 
process.  Two  high-resolution  MRI  volumes  were  registered,  and  they  are  down  sampled  by  1/4  along  each  linear  dimension,  giving 
a  distance  between  voxel  centers  of  ~  5.5  mm.  From  the  optimal  parameters,  we  computed  the  similarity  of  the  simulated  iMRI  and 
MRI  images  as  a  function  of  translations  along  the  coronal  (anterior-posterior)  axis.  Correlation  coefficient  (CC)  is  plotted  in  the 
left  column.  Mutual  information  (MI)  is  plotted  in  the  right  column.  From  the  top  to  bottom,  nearest  neighbor  (NN),  tri-linear  and 
re-normalized  sine  interpolation  methods  are  used  to  obtain  the  floating  images,  respectively.  The  noisey  MI  curves  in  the  right 
column  show  many  local  maxima,  especially  with  sine  interpolation.  CC  curves  in  the  left  column  are  much  smoother  indicating  its 
suitability  for  low  resolution.  Nearest  neighbor  has  a  flat  peak  with  a  width  of  one  voxel  in  both  similarity  curves.  The  unit  of  the 
X-axis  is  voxel.  Images  are  from  volunteer  S2. 
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Figure  2.  Similarity  functions  are  plotted  as  a  function  of  translations  at  full  resolution.  Many  details  are  given  in  the  legend  of 
Figure  1.  Again,  CC  is  plotted  in  the  left  column.  MI  is  plotted  in  the  right  column.  From  the  top  to  bottom,  nearest  neighbor 
(NN),  tri-linear,  and  re-normalized  sine  interpolation  methods  are  used  to  obtain  floating  images,  respectively.  MI  curves  are 
much  peaked  than  CC,  especially  with  sine  and  tri-linear  interpolation.  The  voxel  is  isotropic  with  1 .4  mm  on  a  side.  Image  data 
are  the  same  used  in  Figure  1 . 
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2.4.  Rigid  Body  Registration  Algorithm  with  Special  Features 

The  algorithm  includes  special  features  to  improve  robustness  for  registration  of  MR  prostate  images.1  Suppose  the 
iMRI  image  slice  is  the  reference  image,  the  matching  slice  extracted  from  the  high-resolution  MRI  volume  is  the 
floating  image.  We  use  a  multi-resolution  approach  and  perform  registration  from  low  to  high  resolution.  We  use  CC  at 
the  two  lower  resolutions  because  it  gives  fewer  local  maxima  and  because  it  can  be  calculated  faster  than  MI.  We  use 
MI  at  full  resolution  because  of  its  peaked  surface.  To  avoid  local  maxima,  we  include  a  restarting  feature  where 
registration  is  restarted  with  randomly  perturbed  parameters  obtained  from  a  uniform  distribution  about  the  initial 
transformation  values  at  the  current  resolution  being  used.  The  algorithm  restarts  until  the  absolute  CC  is  above  a 
threshold  of  0.5  as  experimentally  determined  or  the  maximum  number  of  restarts  is  reached.  Absolute  CC  is  used 
rather  than  MI  because  it  has  a  well-defined  range  between  0  and  1  and  because  it  provides  an  independent  check  of  the 
MI  result  at  the  highest  resolution.11 

We  record  all  important  results  following  an  optimization  cycle  including  the  CC  and/or  MI  values  and  the 
transformation  parameters.  At  the  end  of  processing  at  a  lower  resolution,  we  always  select  the  transformation 
parameters  having  the  maximum  CC  value.  We  then  scale  the  translation  parameters  appropriately  and  assign  the  new 
parameters  to  be  initial  values  at  the  next  higher  resolution.  At  the  highest  resolution,  MI  instead  of  CC  is  the  similarity 
measure,  and  we  select  the  final  transformation  parameters  to  be  those  having  the  maximum  MI  value. 

Additional  algorithm  details  are  now  described.  For  registration,  we  use  rigid  body  transformation  (three 
translations  and  three  rotations).  We  chose  tri-linear  interpolation  instead  of  sine  because  it  is  much  faster  and  because 
it  has  comparable  performance,  as  observed  from  Figures  1  and  2.  For  optimization,  we  use  the  downhill  simplex 
method  of  Nelder  and  Mead.17  Optimization  of  similarity  ends  either  when  the  maximum  number  of  calculations  is 
reached  (typically  500)  or  the  fractional  change  in  the  similarity  function  is  smaller  than  a  tolerance  (typically  0.001). 
The  input  MRI  volume  is  a  3D  MR  acquisition  giving  256  x  256  x  128  nearly  isotropic  voxels  over  a  field  of  view 
covering  the  whole  pelvis.  We  create  isotropic  voxels  of  about  1.4  mm  on  a  side  using  3D  linear  interpolation.  We  use 
IDL  (Interactive  Data  Language,  Research  System  Inc.,  Boulder,  CO.)  as  the  programming  language.  We  use  an  initial 
guess  assuming  an  identity  transformation,  i.e.,  all  initial  translation  and  rotation  parameters  are  zero,  because  the 
patient  is  normally  oriented  approximately  the  same  way  from  one  scan  to  the  next.  We  set  the  maximum  numbers  of 
restarts  at  10,  5,  and  3,  from  low  to  high  resolution,  respectively. 

2.5.  Simulation  of  iMRI  Slice  Images 

We  used  high-resolution  MRI  volumes  to  simulate  iMRI  images  by  creating  thick  slices  and  adding  noise  and  receive 
coil  inhomogeneity.11  Clinically,  we  typically  use  an  iMRI  slice  thickness  of  4.0  -  6.0  mm.  We  averaged  3  slices  1.4 
mm  thick  to  create  a  4.2  mm  thick  slice. 

We  added  noise  to  the  simulated  iMRI  image.  MR  noise  is  described  by  the  Rician  distribution,18  but  at  reasonably 
high  signal  values,  the  noise  is  accurately  approximated  with  Gaussian  white  noise.19  We  measured  typical  SNR  on  our 
open  magnet  system  using  a  homogenous  phantom  and  methods  described  elsewhere.20,21  We  then  added  Gaussian 
noise  to  the  simulated  iMRI  slice  images  either  to  match  the  measured  SNR  or  to  give  much  greater  noise  to  further 
stress  registration.  We  report  noise  experiments  using  the  SNR  of  the  simulated  slice  images. 
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2.6.  Registration  Evaluation 

Our  standard  evaluation  method  was  to  compare  SV  and  W  registration.  Since  this  relies  on  W  registration  accuracy, 
we  now  review  our  previous  results.1  For  volume  pairs  acquired  over  a  short  time  span  from  a  supine  subject  with  legs 
flat  on  the  table,  prostates  were  well  aligned  and  prostate  centroid  displacements  were  typically  <  1  mm.  The  registra¬ 
tion  accuracy  as  determined  from  displacements  of  pelvic  bony  landmarks  was  1.6  ±0.2  mm.  This  error  might  be 
overestimated  because  it  includes  the  uncertainty  of  locating  the  bony  landmarks.  The  centroid  error  was  slightly 
smaller  because  the  prostate  was  at  the  volume  center  and  rotation  errors  had  less  effect.  From  our  success  with  VV 
prostate  registration,  we  decided  that  we  could  obtain  SV  accuracy  by  comparing  to  VV  registrations  for  those  volume 
pairs  having  low  W  registration  error. 

To  compare  SV  and  W  registration,  we  defined  a  rectangular  volume  of  interest  (VOI)  just  covering  the  prostate 
and  calculated  voxel  displacements  between  the  two  registrations.  To  voxels  within  the  VOI,  we  applied  the 
transformations  obtained  by  W  and  by  SV  registrations.  We  then  calculated  the  3D  displacements  between  the 
transformed  voxels.  The  mean  voxel  distance  was  used  as  our  metric  of  SV  registration  error.  For  the  evaluation  of 
algorithm  robustness,  we  defined  the  SV  registration  as  being  successful  when  the  3D  displacement  was  less  than  2.0 
mm. 


We  used  the  multiple  visualization  features  of  RegViz,  a  program  created  in  IDL  in  our  laboratory,  to  visually 
evaluate  registration  results.  First,  we  manually  segmented  prostate  boundaries  in  image  slices  and  copied  them  to 
corresponding  slices  from  the  other  volume.  This  enabled  visual  determination  of  the  overlap  of  prostate  boundaries 
over  the  entire  volume.  We  applied  the  same  method  to  evaluate  pelvic  registration.  Second,  color  overlay  displays 
were  used  to  evaluate  overlap  of  structures.  One  image  was  rendered  in  gray  and  the  other  in  the  “hot  -  iron”  color 
scheme  available  in  IDL.  To  visualize  potential  differences,  it  was  quite  useful  to  interactively  change  the  contribution 
of  each  image  using  the  transparency  scale.  Third,  we  used  a  sector  display,  which  divided  the  reference  and  registered 
images  into  rectangular  sectors  and  created  an  output  image  by  alternating  sectors  from  the  two  input  images.  Even 
subtle  shifts  of  edges  could  be  clearly  seen.1 


3.  EXPERIMENTAL  METHODS 


3.1.  Image  Acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system  (Magnetom  Symphony,  Siemens  Medical  Systems, 
Erlangen,  Germany).  An  8-element  phased  array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a  uniform 
sensitivity.  Typically  two  anterior  and  two  posterior  elements  were  enabled  for  signal  acquisition.  We  used  two 
different  MR  sequences.  First,  we  used  a  3D  RF  spoiled  gradient  echo  steady  state  pulse  sequence  (FLASH)  with 
TR/TE/flip  parameters  of  12/5.0/60  which  give  256  x  256  x  128  voxels  over  a  330  x  330  x  256-mm  field  of  view  (FOV) 
to  yield  1.3  x  1.3  x  2.0-mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices.  The  acquisition  time  was 
5.6  min.  This  sequence  was  good  for  pelvic  imaging  but  was  not  ideal  for  the  prostate.  Second,  we  used  a  3D  rapid 
gradient  echo  sequence  (PSIF)  designed  to  acquire  the  spin-echo  component  of  the  steady  state  response,  rather  then  the 
free  induction  decay.  The  spin  echo  component  formed  immediately  prior  to  the  RF  pulse,  and  it  was  shifted  toward  the 
prior  RF  pulse  through  appropriate  gradient  waveform  design.  The  sequence  with  9.4/5.0/60  (TR/TE/flip)  yielded 
160  x  256  x  128  voxels  over  a  219  x  350  x  192-mm  rectangular  FOV  and  1.4  x  1.4  x  1.5-mm  voxels  oriented  to  give  the 
highest  resolution  for  transverse  slices.  There  was  over  sampling  at  31%  in  the  slice  direction  to  reduce  aliasing 
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artifacts.  The  acquisition  time  was  4.3  min.  Most  often,  we  used  the  second  sequence,  which  gave  excellent  image 
contrast  for  the  prostate  and  its  surroundings. 

3.2.  Imaging  Experiments 

We  acquired  high-resolution  MRI  volumes  from  three  volunteers.  For  each  volunteer,  three  image  volumes  were 
obtained  with  an  imaging  session.  Each  volume  was  acquired  with  compatible  conditions.  Volunteers  laid  supine  with 
legs  flat  similar  to  the  position  in  routine  MR  scanning.  Between  volume  acquisitions,  volunteers  got  off  the  MRI  table, 
stretched,  and  walked  around  to  ensure  that  they  would  assume  a  different  position  when  they  laid  back  on  the  table. 
The  coil  array  was  centered  on  the  prostate.  All  images  of  a  volunteer  were  acquired  with  the  same  MRI  acquisition 
parameters.  In  total,  there  are  9  pairs  of  high-resolution  MRI  volumes  for  registration. 


4.  RESULTS  AND  DISCUSSION 


4.1.  Slice  Orientation  and  Position 

As  described  in  Section  2,  we  obtained  relatively  low  noise,  high-resolution  MR  images  and  simulated  SV  registration 
results.  These  data  sets  allowed  us  to  test  effects  of  noise  and  receive  coil  inhomogeneity  in  a  controlled  fashion.  And, 
because  we  had  substantial  previous  experience  showing  the  accuracy  of  W  registration  under  comparable  conditions, 
we  could  easily  determine  SV  error  by  comparing  results  to  W  registration. 

We  determined  SV  registration  results  for  slices  near  the  prostate  in  the  three  standard  orthogonal  orientations. 
Comparing  to  VV,  mean  and  standard  deviation  registration  errors  across  9  volume  pairs  and  45  SV  registration 
experiments  were  0.4  mm  ±  0.2  mm,  0.5  mm  ±  0.2  mm,  and  2.6  mm  ±  1 .6  mm  for  transverse,  coronal  and  sagittal  slices 
covering  the  prostate,  respectively.  Transverse  slices  worked  best  because  they  contain  many  relatively  rigid  anatomical 
structures.  Coronal  slices  worked  next  best.  Sagittal  slices  gave  the  largest  error  because  they  contained  a  large  portion 
of  the  deformable  bladder  and  rectum.  We  further  found  that  transverse  slices  centered  on  the  prostate  produced  better 
results  than  those  above  or  below  the  prostate.  Image  slices  above  included  the  deformable  bladder  that  could  give  an 
inconsistent  structure  from  one  volume  to  the  next.  Image  slices  below  the  prostate  mainly  contained  muscle  and  fatty 
regions  from  the  hips  that  could  deform,  again  giving  inconsistent  image  data. 

4.2.  Noise  Level 

The  typical  iMRI  SNR  under  clinical  conditions  is  about  25.  Even  when  noise  much  exceeded  this  normal  situation, 
registration  results  were  quite  good.  A  100%  success  rate  was  achieved  with  an  acceptance  criterion  of  <  2.0  mm  even 
when  SNR  was  as  bad  as  10. 

4.3.  Robustness 

The  registration  algorithm  was  quite  robust  for  transverse  slices  covering  the  prostate.  Using  the  nine  volume  pairs  from 
three  volunteers,  the  algorithm  never  failed  for  any  transverse  slice  covering  the  prostate.  In  addition,  the  final 
registration  result  was  insensitive  to  initial  guesses  within  a  large  range,  [-60,  +60]  mm  for  translations  and  [-20,  +20] 
degrees  for  rotations.  With  the  restarting  algorithm,  we  even  successfully  registered  slices  as  much  as  80  mm  from  the 
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optimum.  This  working  range  is  probably  sufficient  for  clinical  applications  where  we  can  ensure  good  starting  values. 
Using  the  pelvic  bones  as  markers  and  needle  localization  methods,22  we  should  be  able  to  position  the  prostate  within 
about  ±  20  mm.  In  addition,  the  patient  normally  lies  supine  in  the  MR  bed  with  very  little  rotation  (<  ±  5  degrees). 

4.4.  Comparison  of  Different  Interpolation  Methods 

Computation  time  and  registration  accuracy  are  two  main  factors  to  consider  when  choosing  interpolation  methods. 
Using  tri-linear  interpolation,  the  time  for  an  SV  registration  was  typically  about  5  sec  on  a  Pentium  IV,  1.8  GHz  CPU, 
with  1Gbyte  of  memory.  If  the  re-normalized  sine  interpolation  method  was  used,  the  time  was  ~  10  min,  a  duration  not 
acceptable  for  our  application.  The  algorithm  was  written  in  IDL  and  could  probably  be  made  faster  in  a  lower  level 
language  such  as  C.  We  did  not  use  nearest  neighbor  because  of  the  obvious  insufficient  accuracy  as  induced  from  its 
flat  peak  of  the  similarity  curves  in  Figure  2.  A  call  to  the  Simplex  optimization  typically  resulted  in  50  to  105 
similarity  evaluations  before  the  tolerance  value  (0.001)  was  reached. 

4.5.  Comparison  of  Three  Alignment  Techniques 

When  registering  iMRI  image  slices  with  high-resolution  MR  volumes,  we  compared  three  alignment  techniques: 
(1)  an  iMRI  thick  slice  to  an  MR  thin  slice,  (2)  an  iMRI  thick  slice  to  the  average  of  three  MR  thin  slices,  and  (3)  three 
copies  of  the  iMRI  thick  slice  to  three  MR  thin  slices.  The  second  and  third  methods  were  tested  because  they  were 
thought  to  better  match  the  resolution  of  the  high  resolution  volume  to  the  the  thick  slide.  There  were  no  significant 
differences  (voxel  displacements  <0.1  mm)  among  the  three  methods.  Hence,  we  used  the  first  method  because  of  its 
simplicity. 


5.  CONCLUSION 

The  slice  to  volume  registration  algorithm  is  quite  robust  for  transverse  slice  images  covering  the  prostate.  Since  live¬ 
time  iMRI  images  are  used  for  guidance  and  registered  images  are  used  for  adjunctive  information,  the  registration 
accuracy  is  very  probably  adequate.  It  is  quite  feasible  to  include  previously  acquired  high-resolution  MRI  or  nuclear 
images  into  iMRI -guided  treatment  procedures.  We  are  beginning  to  explore  this  application  in  animal  experiments. 
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Abstract  -  We  are  investigating  automatic  image  registration 
methods  that  can  be  used  for  interventional  magnetic  resonance 
imaging  (iMRI)  guided  radiofrequency  (RF)  thermal  ablation 
of  prostate  cancer.  We  tested  the  ability  of  slice-to-volume 
registration  between  iMRI  slice  images  and  high-resolution 
MRI  volumes.  Images  were  acquired  from  a  conventional  1.5  T 
and  an  interventional  0.2  T  MRI  system.  We  evaluated  the 
registration  quality  by  calculating  3D  displacement  on  a  voxel- 
by-voxel  basis  over  a  volume  of  interest  between  slice-to-volume 
registration  and  volume-to-volume  registration  that  was 
previously  shown  to  be  quite  accurate.  Visual  inspections  such 
as  color  overlay  and  contour  overlap  were  also  used  for 
registration  evaluation.  More  than  300  registration  experiments 
were  performed  on  MR  images  of  volunteers.  Results  showed 
that  the  registration  was  quite  robust  and  accurate  (<  2  mm)  for 
the  transverse  images  covering  the  prostate. 

Keywords:  medical  imaging,  image  registration,  interventional 
magnetic  resonance  imaging  (iMRI),  prostate  cancer. 

I.  Introduction 

We  currently  use  interventional  MRI  as  guidance  for  RF 
thermal  ablation  of  abdominal  cancer.  In  the  case  of  the 
prostate,  the  tumor  is  not  reliably  identified  with  MRI.  Other 
functional  imaging  techniques  are  required.  Potential 
methods  include  SPECT  imaging  and  MR  spectroscopy.  To 
incorporate  the  functional  images  with  interventional  MRI 
tumor  targeting,  one  can  first  register  the  low-resolution 
functional  images  with  high-resolution  MR  volumes.  Then 
by  registering  high-resolution  MR  volume  with  nearly  real¬ 
time  iMRI  acquisitions,  we  can  map  both  the  functional  data 
and  high-resolution  anatomic  information  to  interventional 
images  for  improved  tumor  targeting.  From  our  previous 
success  with  MR  volume-to-volume  (W)  registration  [1] 
and  slice-to-volume  (SV)  registration  using  simulated  iMRI 
images  [2],  we  are  further  testing  the  SV  registration  ability 
using  real  iMRI  images  in  this  study. 

II.  Materials  and  methods 

We  acquired  high-resolution  MR  volumes  from  three 
volunteers  using  a  3D  rapid  gradient  echo  sequence  (PSIF) 
with  9.4/5.0/60  (TR/TE/FA)  and  a  conventional  1.5  T  MR 
system  (Siemens  Magnetom  Symphony, Erlangen,  Germany). 
Volunteers  laid  supine  similar  to  the  diagnostic  position  in 
routine  MR  scanning.  Two  volumes  were  acquired  for  each 
volunteer.  Volume  images  were  normally  256x256x128  with 
the  voxel  size  of  1 .4x1 .4x1 .5-mm. 

We  also  acquired  images  from  the  same  volunteers  using  a 
clinical  0.2  T  C-arm  interventional  MR  system  (Siemens 
Open  Symphony,  Erlangen,  Germany).  Volunteers  were 
supine  with  the  legs  supported  similar  to  the  lithotomy 
position  used  in  prostate  therapies.  This  position  should 
provide  access  for  needle  insertion  in  brachytherapy  or  RF 
thermal  ablation.  We  used  a  3D  PSIF  with  25/13/60 
(TR/TE/FA)  for  volume  acquisitions  and  2D  PSIF  with 
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15.2/7.4/45  (TR/TE/FA)  for  slice  acquisitions.  Slice  images 
were  128x128  with  pixel  size  of  2.8x2.8-mm  and  with 
effective  slice  thickness  of  5  mm.  For  each  volunteer,  we 
acquired  2  volumes  and  25  slices  covering  the  prostate.  They 
included  15  transverse,  5  coronal,  and  5  sagittal  slices.  We 
called  these  images  as  iMRI  volume  or  real  iMRI  slices.  We 
call  them  real  iMRI  slices  to  differentiate  from  previous 
experiments  on  simulated  iMRI  slices. 

We  performed  registration  experiments  using  an  automatic 
algorithm  with  special  features  important  for  robustness  for 
MR  pelvic  images  [1].  First,  we  performed  W  registration 
between  high-resolution  MRI  and  iMRI  volumes.  Second, 
we  extracted  slices  from  an  iMRI  volume  and  registered  the 
slices  with  the  high-resolution  MRI  volume.  We  evaluated 
SV  registrations  by  calculating  3D  displacement  on  a  voxel- 
by-voxel  basis  over  a  volume  of  interest  (VOI)  between  SV 
and  W  registration  [2].  Third,  we  tested  the  SV  registration 
between  real  iMRI  slices  and  high-resolution  MRI  volumes. 
We  used  visual  inspections  such  as  color  overlay  and  contour 
overlap  to  evaluate  the  SV  registration  quality. 

III.  Results 

We  performed  over  300  registration  experiments.  The  W 
registration  of  MRI  and  iMRI  volumes  is  quite  accurate  with 
prostate  centroid  typically  less  than  1  mm.  As  compared  to 
the  W  results,  the  VOI  displacement  of  SV  registration  is 
less  than  2  mm  for  all  transverse  slices  covering  the  prostate. 
Most  coronal  slices  (86%)  gave  an  accuracy  of  less  than 
2.5  mm.  Nearly  half  of  sagittal  slices  (46%)  gave  an 
accuracy  of  less  than  5  mm.  This  suggested  that  transverse 
slices  were  the  best.  Inspection  indicates  that  bladder  and/or 
rectal  filling  interfered  with  registration  of  coronal  and 
sagittal  slices  because  of  deformation.  Visual  inspection  of 
SV  registration  between  iMRI  slices  and  high-resolution 
MRI  volumes  demonstrated  similar  results. 

IV.  Conclusion 

It  is  quite  feasible  to  use  SV  image  registration  to  aid  the 
application  of  interventional  MRI-guided  RF  thermal 
ablation  of  prostate  cancer.  We  are  beginning  to  explore 
these  applications  in  dog  experiments. 
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ABSTRACT 

We  are  investigating  interventional  MRI  guided  radiofrequency  (RF)  thermal  ablation  for  the  minimally  invasive 
treatment  of  prostate  cancer.  Among  many  potential  applications  of  registration,  we  wish  to  compare  registered  MR 
images  acquired  before  and  immediately  after  RF  ablation  in  order  to  determine  whether  a  tumor  is  adequately  treated. 
Warping  registration  is  desired  to  correct  for  potential  deformations  of  the  pelvic  region  and  movement  of  the  prostate. 
We  created  a  two-step,  three-dimensional  (3D)  registration  algorithm  using  mutual  information  and  thin  plate  spline 
(TPS)  warping  for  MR  images.  First,  automatic  rigid  body  registration  was  used  to  capture  the  global  transformation. 
Second,  local  warping  registration  was  applied.  Interactively  placed  control  points  were  automatically  optimized  by 
maximizing  the  mutual  information  of  corresponding  voxels  in  small  volumes  of  interest  and  by  using  a  3D  TPS  to 
express  the  deformation  throughout  the  image  volume.  Images  were  acquired  from  healthy  volunteers  in  different 
conditions  simulating  potential  applications.  A  variety  of  evaluation  methods  showed  that  warping  consistently  improved 
registration  for  volume  pairs  whenever  patient  position  or  condition  was  purposely  changed  between  acquisitions.  A  TPS 
transformation  based  on  180  control  points  generated  excellent  warping  throughout  the  pelvis  following  rigid  body 
registration.  The  prostate  centroid  displacement  for  a  typical  volume  pair  was  reduced  from  3.4  mm  to  0.6  mm  when 
warping  was  added. 

Keywords:  Warping  image  registration,  medical  imaging,  interventional  magnetic  resonance  imaging  (iMRI),  mutual 
information,  thin  plate  spline,  prostate  cancer. 


1.  INTRODUCTION 

We  are  investigating  three-dimensional  (3D)  image  registration  to  be  used  in  applications  of  prostate  cancer  diagnosis, 
staging,  and  therapy.  In  particular,  we  are  interested  in  applications  related  to  the  minimally  invasive  interventional 
magnetic  resonance  imaging  (iMRI)  guided  treatment  of  prostate  cancer.  At  our  institution,  we  currently  use  iMRI  on  a 
low-field  open  magnet  system  to  guide  radiofrequency  (RF)  thermal  ablation  of  abdominal  cancer,1,2  and  we  are 
investigating  this  method  for  prostate  cancer  treatment.  A  unique  feature  of  iMRI-guided  thermal  ablation  is  that  therapy 
can  be  monitored  with  MR  either  by  acquiring  images  of  the  thermally  induced  lesion  or  by  measuring  temperature.  In 
addition,  MR  imaging  of  the  prostate  is  desirable  because  it  more  accurately  delineates  the  prostate  than  does  CT,3,4 
which  can  overestimate  the  prostate  volume,5,6  and  ultrasound,  which  has  a  tendency  to  underestimate  the  extent  of 
lesions.7 

Several  applications  in  prostate  imaging  require  registration.  First,  comparison  of  registered  MR  images  acquired 
before  and  immediately  after  RF  ablation  can  be  used  to  determine  whether  a  tumor  is  adequately  treated.  This  is 
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particularly  helpful  in  instances  where  the  edematous  response  to  treatment  can  be  confused  with  a  highly  perfused 
tumor.  Second,  registration  of  serial  examinations  can  be  used  to  follow  regression/progression  of  tumor.  Third, 
incorporating  the  functional,  biochemical  images  into  the  iMRI  paradigm  will  aid  image-guided  treatments,  as  recently 
reported  by  us.8,9 

Several  reports  describe  methods  for  registration  in  the  pelvis  or  prostate.10'15  These  methods  required  either 
segmentation  or  visual  identification  of  structures  and  they  were  based  on  rigid  body  transformation.  We  recently 
reported  a  mutual  information  rigid  body  transformation  method  for  prostate  registration.16  For  volume  pairs  acquired 
over  a  short  time  span  from  a  supine  subject  with  legs  flat  on  the  table,  registration  accuracy  of  both  prostate  centroids 
(typically  <  1  mm)  and  bony  landmarks  (average  1.6  mm)  was  on  the  order  of  a  voxel  (~  1.4  mm).  We  obtained 
somewhat  larger  prostate  registration  errors  of  about  3.0  mm  when  volume  pairs  were  obtained  under  very  different 
conditions,  e.g.,  legs  flat  and  legs  raised,  or  with  and  without  bladder  or  rectal  filling.  Rigid  body  registration  of  the 
pelvis  cannot  follow  prostate  movements  due  to  changes  in  the  postures  of  legs  and  deformation  of  the  bladder  and 
rectum,  as  reported  by  us  16  and  others.14,17 

There  are  a  number  of  reports  on  warping  registration.18'20  These  applications  were  mainly  on  the  brain  and  breast; 
and  far  few  described  results  for  the  abdomen.21,22  For  our  application,  we  are  developing  voxel-based  warping 
registration  in  the  pelvis  and  prostate. 

There  are  challenges  to  the  pelvis  and  prostate  registration.  First,  pelvic  regions  can  change  shape  significantly, 
unlike  the  brain  to  which  registration  has  been  most  often  applied.  Different  patient  positions  such  as  displacement  of  the 
legs  can  cause  the  movement  and  deformation  of  internal  organs.  Second,  the  normal  prostate  is  a  small  organ  that  when 
healthy  measures  only  about  3.8  cm  in  its  widest  dimension  transversely  across  the  base.23  Third,  the  prostate  might 
move  relative  to  the  pelvic  bones  due  to  changes  in  bladder  and  rectal  filling.14,17 

In  the  present  study,  we  perform  experiments  to  determine  if  warping  can  improve  registration  of  pelvic  MR.  High 
quality,  3D  MR  image  volumes  from  a  commercially  available  1.5  T  system  are  used  to  determine  the  best  possible 
results.  We  examine  conditions  found  in  the  potential  clinical  applications  described  previously.  We  will  qualitatively 
and  quantitatively  compare  results  of  warping  and  rigid  body  registration  using  9  volume  pairs  from  three  volunteers. 


2.  REGISTRATION  ALGORITHM 


2.1.  Similarity  Measurements 


We  used  two  similarity  measures,  mutual  information  and  correlation  coefficient  (CC),  in  our  registration.  One  volume  R 
is  the  reference ,  and  the  other  F  is  floating.  Their  mutual  information  MI(R ,  F)  is  given  below.24 


r,f 


PrAtJ) 
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The  joint  probability  pRF(r,f)  and  the  marginal  probabilities  pR  (V)  of  the  reference  image  and  p F  (/)  of  the 
floating  image,  can  be  estimated  from  the  normalized  joint  intensity  histograms.  When  two  images  are  geometrically 
aligned,  MI  is  maximal.24  The  correlation  coefficient  CC(R,F)  is  given  below.25 
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Here  i?(r),  F(f)  denote  the  average  intensities  of  the  reference  and  floating  volumes  and  the  summation  includes  all 
voxels  within  the  overlap  of  both  volumes. 


2.2.  Rigid  Body  Registration  Algorithm  with  Special  Features 

The  rigid  body  registration  algorithm  includes  special  features  to  improve  robustness  for  MR  pelvic  images.  Reference  is 
matched  to  the  reformatted  volume  obtained  by  transforming  the  floating  volume.  We  use  a  multi-resolution  approach 
and  perform  registration  from  low  to  high  resolution.  At  low  resolution,  we  resample  both  images  at  1/4  or  1/2  number  of 
voxels  along  each  linear  dimension,  respectively.  A  simplex  algorithm  varies  the  six  rigid  body  transformation 
parameters  (three  translations  and  three  angles)  to  optimize  the  similarity  measure.26  We  use  CC  at  the  two  lower 
resolutions  because  it  gives  fewer  local  maximums  8,9,16  and  because  it  can  be  calculated  faster  than  MI.  We  use  MI  at 
full  resolution  because  the  peaked  similarity  function  gives  a  more  precise  solution  than  CC.9,16  To  avoid  local 
maximums,  we  include  a  restarting  feature  where  registration  is  restarted  with  randomly  perturbed  parameters.  The 
algorithm  restarts  until  the  absolute  CC  is  above  an  experimentally  determined  threshold  or  the  maximum  number  of 
restarts  is  reached.  Absolute  CC  is  used  for  the  restart  test  rather  than  MI  because  CC  has  fewer  problems  with  local  and 
incorrect  global  maximums  at  low  resolution.9,16 

We  record  all  important  results  following  an  optimization  cycle  including  the  CC  and/or  MI  values,  the  number  of 
restarts,  and  the  transformation  parameters.  At  the  end  of  processing  at  a  lower  resolution,  we  always  select  the 
transform-ation  parameters  having  the  maximum  CC  value.  We  then  scale  the  translation  parameters  appropriately  and 
assign  the  new  parameters  to  be  initial  values  at  the  next  higher  resolution.  At  the  highest  resolution,  we  select  the  final 
transformation  parameters  to  be  those  with  the  maximum  MI  value. 

Typical  parameter  values  are  now  described.  We  use  an  initial  guess  at  the  lowest  resolution  of  all  zeros  because  the 
patient  is  normally  oriented  approximately  the  same  way  from  one  scan  to  the  next.  Based  on  experience,  we  set  the  CC 
thresholds  at  0.50,  0.55,  and  0.65,  and  the  maximum  numbers  of  restarts  at  10,  5,  and  3,  from  low  to  high  resolution, 
respectively. 

2.3.  Warping  Registration  Using  Optimized  Control  Points 

The  warping  registration  algorithm  includes  three  major  steps:  control  point  (CP)  selection,  control  point  optimization, 
and  thin  plate  spline  (TPS)  warping.  Again,  the  unchanging  volume  is  the  reference ,  and  the  one  to  be  warped  is  floating . 

The  manual  selection  of  CP's  is  an  important  step.  We  used  RegViz ,  a  program  written  in  IDL  (Interactive  Data 
Language,  Research  System  Inc.,  Boulder,  CO)  and  created  in  our  laboratory  for  visualizing  and  analyzing  image 
volumes.  Following  rigid  body  registration,  the  aligned  two  volumes  are  displayed  in  two  rows  slice-by-slice. 
Corresponding  CP’s  in  the  two  volumes  are  placed  using  a  cursor,  and  sometimes  they  are  in  different  image  slices.  The 
3D  coordinates  are  automatically  stored  in  a  file.  Because  of  the  optimization  that  occurs  later,  the  correspondence  can  be 
up  to  15  mm  or  ~  10  voxels  in  error.  More  experience  with  CP  selection  is  described  in  Results.  Typically,  we  used  180 
CP’s  for  a  volume  with  256  x  256  x  140  isotropic  voxels. 

The  next  step  of  the  warping  algorithm  is  the  CP  optimization.  We  define  a  small  cubic  volume  of  interest  (VOI) 
centered  at  each  CP.  The  VOI  can  be  16  or  64  voxels  on  a  side.  A  simplex  optimization  algorithm  varies  the  x,  y,  and  z 
transformation  parameters  of  the  floating  VOI  until  the  mutual  information  with  the  reference  VOI  is  optimized.  Each 
control  point  is  optimized  independently. 
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The  final  major  step  is  to  warp  the  floating  volume  using  the  optimal  CP's  coordinates  to  establish  a  TPS 
transformation.  We  extended  the  two-dimensional  TPS  transformation  method  reported  by  Bookstein27  to  three 
dimensions  in  a  manner  similar  to  that  of  Davis  et  al.19 


2.4.  Additional  Details 

There  are  several  preprocessing  details.  The  input  MRI  volume  is  a  3D  MR  acquisition  giving  256  x  256  x  128  nearly 
isotropic  voxels  over  a  field  of  view  covering  the  whole  pelvis.  We  create  isotropic  voxels  of  about  1.4  mm  on  a  side 
using  3D  linear  interpolation.  From  the  top  and  bottom  of  the  volume,  we  optionally  crop  transverse  slices  that  are  over 
35  mm  away  from  the  prostate  rim.  Cropping  is  done  to  remove  slices  having  reduced  brightness  due  to  sensitivity  fall 
off  from  the  receiver  coils  and/or  artifacts  from  a  small  field  of  view.  For  both  rigid  body  registration  and  VOI 
optimization,  we  use  trilinear  interpolation.  Optimization  of  similarity  ends  either  when  the  maximum  number  of 
calculations  is  reached  (typically  500)  or  the  fractional  change  in  the  similarity  function  is  smaller  than  a  tolerance 
(typically  0.001).  We  use  IDL  as  the  programming  language. 


3.  EXPERIMENTAL  METHODS 


3.1.  Image  Acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system  (Magnetom  Symphony,  Siemens  Medical  Systems, 
Erlangen,  Germany).  An  8-element  phased  array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a  uniform 
sensitivity.  Typically  two  anterior  and  two  posterior  elements  were  enabled  for  signal  acquisition.  We  used  two  different 
MR  sequences.  First,  we  used  a  3D  RF  spoiled  gradient  echo  steady  state  pulse  sequence  (FLASH)  with  TR/TE/flip 
parameters  of  12/5.0/60  which  give  256  x  256  x  128  voxels  over  a  330  x  330  x  256-mm  field  of  view  (FOV)  to  yield 
1.3  x  1.3  x  2.0-mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices.  The  acquisition  time  was  5.6  min. 
This  sequence  was  good  for  pelvic  imaging  but  was  not  ideal  for  the  prostate.  Second,  we  used  a  3D  rapid  gradient  echo 
sequence  (PSIF)  designed  to  acquire  the  spin-echo  component  of  the  steady  state  response,  rather  then  the  free  induction 
decay.  The  spin  echo  component  formed  immediately  prior  to  the  RF  pulse,  and  it  was  shifted  toward  the  prior  RF  pulse 
through  appropriate  gradient  waveform  design.  The  sequence  with  9.4/5.0/60  (TR/TE/flip)  yielded  160x256x  128 
voxels  over  a  219  x  350  x  192 -mm  rectangular  FOV  and  1.4  x  1.4  x  1.5-mm  voxels  oriented  to  give  the  highest 
resolution  for  transverse  slices.  There  was  over  sampling  at  31%  in  the  slice  direction  to  reduce  aliasing  artifacts.  The 
acquisition  time  was  4.3  min.  Most  often,  we  used  the  second  sequence,  which  gave  excellent  image  contrast  for  the 
prostate  and  its  surroundings. 

3.2.  Imaging  Experiments 

We  acquired  3D  MRI  volume  images  from  three  normal  volunteers  under  a  variety  of  conditions  simulating  anticipated 
conditions  in  diagnostic  and  treatment  applications.  Before  image  acquisition,  each  volunteer  drank  water  and  had  a 
relatively  full  bladder .  In  the  diagnostic  position,  the  subject  laid  supine  throughout  MR  scanning.  In  the  treatment 
position ,  the  subject  was  supine,  and  his  legs  were  supported  at  30°-60°  relative  to  the  horizon  and  separated  in  a  “V” 
with  an  angle  of  60°-90°  between  two  legs.  This  position  should  provide  access  for  needle  insertion  in  brachytherapy  or 
RF  thermal  ablation.  In  some  experiments,  the  subject  micturated  to  create  an  empty  bladder  prior  to  imaging.  For  each 
subject,  image  volumes  were  obtained  on  the  same  day  within  one  imaging  session.  Between  volume  acquisitions, 
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volunteers  got  off  the  MRI  table,  stretched,  and  walked  around  to  ensure  that  they  would  assume  a  different  position 
when  they  laid  back  on  the  table.  All  images  of  a  volunteer  were  acquired  with  the  same  MRI  acquisition  parameters  so 
as  to  ensure  very  similar  gray  values.  In  total,  there  are  3  volumes  for  each  volunteer. 


3.3.  Volume  Pairs  for  Registration 

We  registered  9  volume  pairs  under  different  conditions.  Three  pairs  are  treatment-diagnosis;  three  pairs  are  full-empty 
bladder;  and  three  pairs  are  diagnosis-diagnosis .  Rigid  body  and  warping  registration  were  applied  to  each  of  the  volume 
pairs.  Results  were  evaluated  as  described  next. 


3.4.  Registration  Evaluation 

We  used  the  multiple  visualization  features  of  RegViz  to  visually  evaluate  registration  results.  First,  we  manually 
segmented  prostate  boundaries  in  image  slices  and  copied  them  to  corresponding  slices  from  the  other  volume.  This 
enabled  visual  determination  of  the  overlap  of  prostate  boundaries  over  the  entire  volume.  We  applied  the  same  method 
to  evaluate  pelvic  registration.  Second,  color  overlay  displays  were  used  to  evaluate  overlap  of  structures.  One  image  was 
rendered  in  gray  and  the  other  in  the  “hot  -  iron”  color  scheme  available  in  IDL.  To  visualize  potential  differences,  it  was 
quite  useful  to  interactively  change  the  contribution  of  each  image  using  the  transparency  scale.  Third,  we  used  a  sector 
display,  which  divided  the  reference  and  registered  images  into  rectangular  sectors  and  created  an  output  image  by 
alternating  sectors  from  the  two  input  images.  Even  subtle  shifts  of  edges  could  be  clearly  seen.16 

Voxel  gray  value  measures  were  calculated  as  indicators  of  regitration  quality.  Mutual  information  and  correlation 
coefficient  between  registered  volumes  were  computed.  The  higher  the  MI  values,  the  more  likely  the  two  volumes  are 
well  registered.  Since  volumes  to  be  registered  were  acquired  using  the  same  acquisition  parameters,  high  absolute  CC 
values  were  obtained  when  registration  was  good.20  Because  voxel  intensities  were  comparable,  we  created  difference 
images  and  calculated  statistics  such  as  the  voxel  mean  and  standard  deviation  following  registration. 

Finally,  to  assess  prostate  registration  error,  we  measured  potential  displacements  of  the  3D  centroid  of  the  prostate. 
We  used  RegViz  to  manually  segment  the  prostate  across  all  image  slices  and  calculated  the  3D  centroid.  A  similar 
method  was  used  by  West  et  al.  to  evaluate  rigid  body  registration  of  the  brain.28 


4.  RESULTS  AND  DISCUSSION 


4.1.  Control  Point  Selection 

In  well  over  50  registration  experiments,  we  investigated  the  number  and  placement  of  CP’s  on  warping  registration 
quality.  We  quickly  determined  that  many  CP’s  are  required  in  order  to  get  good  visual  matching  of  our  very  high- 
resolution  MR  images  showing  great  anatomical  detail,  especially  when  large  deformations  occurred  as  in  the  case  of 
diagnosis-treatment.  We  progressively  increased  the  number  of  CP’s  from  15  to  250.  We  found  that  less  than  120  CP's 
did  not  produce  smooth  and  reasonable  warps.  Warping  with  180  CP's  excellently  approximated  the  deformation  of  the 
pelvis  and  internal  organs.  More  than  220  CP's  did  not  give  significant  improvement  but  needed  more  time  for  manual 
selection  and  optimization.  As  a  result  of  this  experience,  we  modified  the  registration  method  to  be  more  suitable  for 
many  CP’s  (see  Section  2). 
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Figure  1.  Control  point  selection  when  images  are  acquired  in  the  treatment  and  diagnostic  positions.  Image  (a)  is  from 
the  reference  volume  acquired  in  the  treatment  position  with  legs  raised.  Image  (b)  is  to  be  warped  and  is  from  the  volume 
acquired  in  the  diagnostic  position  with  the  subject  supine  on  the  table.  Transverse  slices  best  show  the  deformations, 
especially  at  the  legs.  As  described  in  the  text,  control  points  indicated  by  the  white  dots  are  selected  around  the  pelvic 
surface  and  the  prostate.  Each  control  point  is  located  at  one  voxel  but  displayed  much  bigger  for  better  visualization. 
Volumes  are  from  volunteer  S2. 


We  also  determined  that  it  was  necessary  to  place  the  CP’s  in  strategic  locations,  and  some  rules  follow.  For 
registration  of  treatment  and  diagnostic  image  volumes,  most  CP’s  were  selected  using  transverse  slices  because  they 
best  showed  the  pelvic  displacement  when  moving  the  legs  to  the  treatment  position  (Figure  1).  About  30  CP  pairs  were 
placed  near  edge  and  point  features  having  recognizable  correspondence  on  each  of  5-8  transverse  slices  with  a  z  interval 
of  ~  8  mm,  covering  the  entire  prostate  region.  Additionally,  we  placed  about  25  CP's  from  sagittal  slices  because  they 
provided  other  structures  that  can  be  missed  in  the  transverse  images.  It  was  also  important  to  include  CP's  from  organs 
other  than  the  prostate  because  they  constrained  warps.  We  always  placed  CP's  at  critical  regions  such  as  the  prostate 
center,  pelvic  surface,  bladder  border,  and  rectal  walls. 


4.2.  Registration  Quality  of  Warping  and  Rigid  Body  Registration 

In  Figure  2,  we  compare  warping  and  rigid  body  registration  for  a  typical  volume  pair  in  the  treatment  and  diagnostic 
positions.  Using  rigid  body  registration,  there  is  significant  misalignment  throughout  large  regions  in  the  pelvis  (Figure 
2a)  that  is  greatly  reduced  with  warping  (Figure  2b).  Note  that  warping  even  allows  the  outer  surfaces  to  match  well. 
Other  visualization  methods  such  as  two-color  overlays  and  difference  images,  quickly  show  matching  of  structures 
without  segmentation  but  do  not  reproduce  well  on  a  printed  page.  The  prostate  3D  centroid  calculated  from  segmented 
images  displaced  by  only  0.6  mm,  or  0.4  voxels,  following  warping.  Following  rigid  body  registration,  the  prostate  was 
misaligned  with  a  displacement  to  the  posterior  of  ~  3.4  mm  when  in  the  treatment  position 

We  examined  the  effect  of  conditions  such  as  bladder  and  rectal  filling  that  might  change  from  one  imaging  session 
to  the  next  (not  shown).  Contour  overlap  showed  that  warping  registration  perfectly  aligns  the  prostate  while  rigid  body 
does  not.  In  addition,  rigid  body  registration  does  not  align  the  bladder.  With  warping,  the  bladder  perfectly  matches  the 
reference.  Other  visualization  methods  showed  excellent  alignment  of  internal  and  surface  edges.  Difference  images 
show  that  warping  greatly  improves  alignment  of  internal  structures  as  compared  to  rigid  body  registration. 

We  also  examined  volume  pairs  with  both  volumes  acquired  in  the  diagnostic  position  under  comparable  conditions. 
In  all  such  cases,  rigid  body  registration  worked  as  well  as  warping.  There  were  no  noticeable  deformations  in  the  pelvis, 
and  prostate  centroids  typically  displaced  less  than  1.0  mm  between  the  two  registered  volumes. 
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Figure  2.  Comparison  of  warping  and  rigid  body  registration  for  volumes  acquired  in  the  treatment  and  diagnostic 
positions.  Image  (a)  is  from  the  floating  volume  acquired  in  the  diagnostic  position  following  rigid  body  registration. 
Pelvic  contours  manually  segmented  from  the  reference  show  significant  misalignment.  Image  (b)  is  from  the  floating 
volume  acquired  in  the  diagnostic  position  following  warping  registration.  The  misalignment  is  greatly  improved  with 
warping.  Images  are  transverse  slices  from  subject  S2. 
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Figure  3.  Image  statistics  of  absolute  intensity  difference  images  for  rigid  body  and  warping  registration.  The  mean  (a)  and 
standard  deviation  (b)  are  plotted.  See  the  legend  of  Figure  3  for  other  details.  Warping  decreased  the  mean  and  standard 
deviation  in  each  case,  but  the  most  significant  decreases  occurred  in  the  case  of  the  treatment-diagnosis  volume  pairs. 
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4.3.  Quantitative  Evaluation  of  Warping  Registration 


Statistics  of  image  differences  following  rigid  body  and  warping  registration  are  shown  in  Figure  3.  Warping  reduces  the 
absolute  intensity  difference  between  corresponding  voxels  (Figure  3a).  We  did  not  use  non-absolute  differences  as 
evaluation  criteria  because  the  positive  and  negative  differences  can  be  canceled.  The  standard  deviation  of  absolute 
difference  is  also  reduced  (Figure  3b).  We  also  calculated  the  correlation  coefficient  and  mutual  information  values 
between  registered  volumes.  Warping  increased  CC  and  MI  values  in  every  case. 


4.4.  Algorithmic  Robustness  and  Efficiency 

The  rigid  body  algorithm  gives  robust  registration.  Because  of  two  principal  design  features,  the  algorithm  is  quite  robust 
and  accurate  for  volume  pairs  acquired  in  the  same  positions  and  with  comparable  conditions.16  First,  using  both  CC  and 
MI  at  different  resolutions  was  an  important  feature  that  increased  robustness.  CC  gave  fewer  local  minimums  at  low 
resolutions  and  MI  was  more  accurate  at  high  resolution.8,9  Second,  the  restarting  mechanism  was  also  quite  important. 
Without  restarting,  we  found  that  registrations  sometimes  failed  in  cases  of  volumes  with  large  mismatches  and 
significant  deformation.  Even  these  cases  resulted  in  a  proper  solution  when  restarting  was  employed.  The  time  for  rigid 
body  registration,  typically  5  minutes  on  a  Pentium  IV,  1.8  GHz  CPU,  with  1.0  GBytes  of  memory,  could  be  greatly 
improved  with  C  rather  than  IDL. 

The  interactive  warping  registration  algorithm  is  designed  to  be  very  computationally  efficient  for  TPS  warping  with 
hundreds  of  CP’s.  First,  the  optimization  of  small  VOI’s  is  very  fast.  Second,  we  optimized  each  CP  separately  because 
the  optimization  of  three  parameters  (x,  y,  and  z)  is  simple  and  fast.  Third,  we  applied  the  TPS  transformation  once  to  the 
final,  optimal  CP's;  this  saved  considerable  time.  Using  the  same  computer  above,  for  a  volume  with  256x256x140 
voxels  and  180  CP’s,  the  warping  registration  typically  takes  about  15-45  minutes  depending  on  the  VOI  size.  If  we  were 
to  use  optimized  C  code,  the  total  time  for  rigid  body  and  warping  registration  should  reduce  to  within  5  minutes. 


5.  CONCLUSION 

The  mutual  information  warping  registration  works  better  than  rigid  body  registration  whenever  the  subject  position  or 
condition  is  greatly  changed  between  acquisitions.  It  will  probably  be  a  useful  tool  for  many  applications  in  prostate 
diagnosis,  staging,  and  therapy.  It  is  fast  for  hundreds  of  control  points  and  can  be  applied  to  a  variety  of  applications. 
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ABSTRACT 

We  are  investigating  methods  to  register  live-time  interventional  magnetic  resonance  imaging  (iMRI)  slice  images  with  a 
previously  obtained,  high  resolution  MRI  image  volume.  The  immediate  application  is  for  iMRI-guided  treatments  of 
prostate  cancer.  We  created  and  evaluated  a  slice-to-volume  mutual  information  registration  algorithm  for  MR  images  with 
special  features  to  improve  robustness.  Features  included  a  multi-resolution  approach  and  automatic  restarting  to  avoid  local 
minima.  We  acquired  3D  volume  images  from  a  1.5  T  MRI  system  and  simulated  iMRI  images.  To  assess  the  quality  of 
registration,  we  calculated  3D  displacement  on  a  voxel-by-voxel  basis  over  a  volume  of  interest  between  slice-to-volume 
registration  and  volume-to- volume  registrations  that  were  previously  shown  to  be  quite  accurate.  More  than  500  registration 
experiments  were  performed  on  MR  images  of  volunteers.  The  slice-to-volume  registration  algorithm  was  very  robust  for 
transverse  slice  images  covering  the  prostate.  A  100%  success  rate  was  achieved  with  an  acceptance  criterion  of  <  1.0  mm 
displacement  error  over  the  prostate.  Our  automatic  slice-to-volume  mutual  information  registration  algorithm  is  robust  and 
probably  sufficiently  accurate  to  aid  in  the  application  of  iMRI-guided  thermal  ablation  of  prostate  cancer. 

Keywords:  Image  registration,  mutual  information,  magnetic  resonance  imaging  (MRI),  interventional  MRI,  prostate 
cancer,  minimally  invasive  treatment,  thermal  ablation. 


1.  INTRODUCTION 

We  are  investigating  methods  for  registering  slice  images  obtained  on  a  low  field  iMRI  system  to  high-resolution  MR 
volumes  obtained  on  a  traditional  1.5  T  scanner.  Immediate  applications  involve  the  treatment  of  prostate  cancer  as 
described  below.  To  be  useful  for  an  interventional  procedure,  a  registration  method  must  be  automatic,  accurate,  robust,  and 
fast.  Currently,  we  are  investigating  voxel-based  methods  because  of  their  ease  of  use  and  reported  accuracy  for  other 
applications. 

There  are  important  reasons  for  this  investigation.  At  our  institution,  we  have  extensive  experience  with  minimally 
invasive  treatment  of  abdominal  cancer  using  iMRI-guided  radio  frequency  thermal  abaltion. 1,2,3  In  the  case  of  the  prostate, 
the  tumor  is  not  reliably  identified  with  MR.  Hence,  other  functional  imaging  techniques  are  required,  and  potential  methods 
include  SPECT  antibody  imaging  and  MR  spectroscopy.  To  incorporate  the  functional  images  with  iMRI  tumor  targeting, 
one  can  first  register  the  low-resolution  functional  images  with  a  high-resolution  MRI.  Then  by  registering  the  high- 
resolution  MR  volume  with  live-time  iMRI  acquisitions,  we  can  map  both  the  functional  data  and  high-resolution  anatomic 
information  to  live-time  iMRI  images  for  improved  tumor  targeting.  Image  guided  biopsies  are  another  important  application 
of  iMRI  in  those  cases  where  the  tumor  is  more  readily  seen  in  MR  than  in  CT  or  ultrasound.  4,5,6 

Previously  reported  methods  on  slice  to  volume  registration  were  mainly  applied  to  the  brain. 7,8,9  For  the  case  of  volume 
to  volume  registration,  there  are  many  reports  of  accurate  voxel-based  registration  in  the  brain  10,11,12,13  and  abdominal  and 
pelvic  organs.1,14 
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There  are  challenges  to  successful  slice  to  volume  registration  for  the  prostate.  They  include: 

1)  A  single  slice,  or  a  few  slices,  provides  much  less  information  than  an  entire  volume  for  voxel  based  matching. 

2)  There  is  a  low  signal  to  noise  ratio  (SNR)  in  iMRI  because  of  the  emphasis  on  fast  imaging  typically  with  a  low  field 
scanner. 

3)  The  pelvic  region  has  irregular  boundaries  and  can  deform,  unlike  the  brain  to  which  registration  has  been  most  often 
applied.  The  prostate  might  move  relative  to  the  pelvic  bones  due  to  changes  in  rectal  and  bladder  filling  15,16  or 
movement  of  the  patient  for  treatment. 

4)  The  normal  prostate  is  a  small  organ  that  when  healthy  measures  only  about  3.8  cm  in  its  widest  dimension  transversely 
across  the  base,  17  and  the  small  prostate  is  located  below  a  much  larger  bladder  that  can  change  shape  and  size. 

5)  The  non-homogenous  receive  coil  response  can  change  from  one  acquisition  to  next. 

We  are  investigating  the  use  of  voxel-based  registration  for  this  important  application.  We  created  a  mutual  information 

algorithm  modified  to  include  some  features  to  improve  robustness.  We  performed  registration  experiments  under  conditions 

found  with  low  field  open  MR  imaging. 


2.  REGISTRATION  ALGORITHM 


2.1.  Similarity  Measurement 

Two  similarity  measurements,  mutual  information  (MI)  and  correlation  coefficient  (CC),  are  used  in  our  algorithm.  The 
Mi-based  registration  method  is  robust  and  suitable  for  multi-modality  registration,  is  highly  accurate  for  brain 
registration,18  and  is  suitable  for  abdominal  registration  where  there  can  be  deformation.1,14  However,  the  MI  method  has  the 
problem  of  interpolation  artifacts,  which  are  especially  serious  in  the  case  of  down  sampling  in  a  multi-resolution  approach.19 
Fortunately,  CC  produces  fewer  locally  optimum  values  than  does  MI.20  Our  method  combines  both  similarity  measures  to 
use  the  good  attributes  of  each. 

MI  quantifies  interdependency  of  two  variables,  such  as  image  gray  intensities  of  image  A  and  B.  When  A  and  B  are 
exactly  aligned,  MI  is  maximal.18  We  calculate  MI  using  Equations  9  to  12  in  the  report  by  Maes  et  al.18  CC  is  a  measure  of 
the  relation  between  the  statistical  distributions  of  the  two  images.  The  absolute  of  CC  has  the  range  of  0  to  1.  The  higher  the 
absolute  value  of  CC,  the  more  dependence  the  two  images  have.21  We  calculate  CC  using  the  equation  10  in  the  report  by 
Rueckert  et  al.21 

The  algorithm  shown  in  Figure  1  included  special  features  to  improve  robustness  for  registration  of  MR  prostate  images. 
In  the  pseudo-code,  the  iMRI  slice  image  is  the  reference  slice ,  the  slice  image  extracted  from  the  high-resolution  MRI 
volume  is  the  reformatted  slice,  and  the  final  reformatted  slice  is  the  registered  slice.  We  used  a  multi-resolution  approach 
and  performed  registration  from  low  to  high  resolution.  We  used  MI  at  the  highest  resolution  because  it  gave  a  more  robust 
solution,  and  we  used  CC  at  the  lower  resolutions  because  it  gives  few  local  optima  and  because  it  calculates  faster  than  MI. 
We  created  a  method  to  avoid  local  minima  by  restarting  the  registration  with  randomly  perturbed  parameters  obtained  from 
a  uniform  distribution  about  the  very  first  initial  guess.  The  algorithm  restarts  until  the  absolute  CC  between  the  reference 
and  registered  images  is  above  a  threshold  or  the  maximum  number  of  restarts  is  reached.  Absolute  CC  is  used  for  the  restart 
test  rather  than  MI  because  it  has  a  well-defined  range  between  0  and  1  and  because  it  provides  an  independent  check  of  the 
MI  result. 

We  used  rigid  body  transformation  (three  translations  and  three  rotations)  and  trilinear  interpolation  as  described 
previously.1  For  optimization,  we  used  the  downhill  simplex  method  of  Nelder  and  Mead 22  and  the  Powell  method,23  but  we 
prefer  the  former  method  as  described  later.  Optimization  of  similarity  ends  either  when  the  maximum  number  of 
calculations  is  reached  (typically  500)  or  the  fractional  change  in  similarity  function  is  smaller  than  a  tolerance  (typically 
0.001).  Our  very  first  initial  guess  at  the  lowest  resolution  is  all  zeros  for  the  3  displacements  and  3  angles.  Based  on  our 


Proc.  SPIE  Vol.  4319 


54 


Set  an  initial  reformatted_slice  in  high-resolution  MR  volume  and  CCjhresholds 
DO  registration  FROM  lowest  resolution  TO  highest  resolution  BEGIN 
Resample  volume  and  slice  to  %,  lA  or  lull  number  of  voxels  along  linear  dimension 
Initialize  registration_results  and  number_restarts  to  zero  for  restarting  registrations 

REPEAT 

Optimize  similarity  (CC*  or  MI**)  between  the  reference  slice  and  reformatted_slice 

1.  Transform  the  initial  reformatted_slice  and  interpolate  to  get  a  new  reformatted_slice 

2.  Calculate  similarity  between  the  reference  slice  and  the  new  reformatted_slice 

3.  Repeat  1  and  2  until  meeting  function  tolerance  or  maximum  iteration  number 
Calculate  CC  between  the  reference  slice  and  the  optimal  reformatted_slice 
Record  CC,  MI**  values  and  transformation  parameters  to  registration_results 
Perturb  the  initial  transformation  parameters: 

initial_transformation_parameters  =  initial_transformation_parameters  +  random  •  factor 
Trace  the  number  of  restarts:  number_restarts  =  number_restarts  +  1 
UNTIL  (CC  >  CC_threshold)  OR  (number_restarts  >  maximum_restarts) 

Select  the  final  transformation_parameters  based  on  CC*  or  MI**  among  registration_results 
Scale  the  parameters  and  assign  to  initial_transformation_parameters  of  next  higher  resolution 
END 


Figure  1 .  Registration  algorithm.  Capital  bold  words  are  computer  language.  The  outer  loop  from  DO  to  END 
gives  the  multi-resolution  approach.  The  inter  loop  from  REPEAT  to  UNTIL  is  for  restarting  registration. 
Registration^results  and  number_restarts  are  used  to  store  temporary  values  in  the  program.  See  text  for  details. 

*  CC  is  used  at  lower  resolutions,  lA  or  Vi  number  of  voxels. 

**  MI  is  used  only  at  high  resolution,  full  number  of  voxels. 

experience,  we  set  the  CC  thresholds  at  0.60,  0.65  and  0.70,  and  the  maximum  numbers  of  restarts  at  25,  15,  and  5,  from  low 
to  high  resolution,  respectively. 

In  Figure  1,  there  are  several  differences  between  low  and  high  resolutions.  At  lower  resolutions,  we  resample  the 
volume  and  slice  to  1/4  or  1/2  size  along  the  linear  dimension.  We  optimize  the  CC  between  reference  and  reformatted  slices 
to  obtain  optimal  transformation  parameters.  We  record  important  results  such  as  all  CC  optimized  values,  the  number  of 
restarts,  and  the  transformation  parameters  following  an  optimization.  At  the  end  of  either  of  these  resolutions,  we  select  the 
transformation  parameters  that  have  the  maximum  CC  value.  We  then  scale  the  parameters  and  assign  them  to  be  initial 
values  at  the  next  higher  resolution.  At  the  highest  resolution,  MI  instead  of  CC  is  chosen  to  be  the  similarity  measurement 
method,  and  we  select  the  final  transformation  parameters  with  maximum  MI  instead  of  CC. 

There  are  several  preprocessing  details.  The  input  MRI  volumes  are  256  x  256  x  128  voxels  that  have  an  almost  isotropic 
size  over  a  field  of  view  covering  the  whole  pelvis.  Isotropic  voxels  are  created  using  3D  linear  interpolation  or  higher  order 
interpolation  methods.1 


3.  EXPERIMENTAL  METHODS 


3.1.  Data  Acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system  (Magnetom  Symphony,  Siemens  Medical  Systems 
in  Erlangen,  Germany).  An  8-element  phased  array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a  uniform 
sensitivity.  Typically  two  anterior  and  two  posterior  elements  were  enabled  for  signal  acquisition.  We  used  a  3D  PSIF 
sequence  with  9.4/5.0/60  (TR/TE/flip)  yields  160  x  256  x  128  voxels  over  a  219  x  350  x  192-mm  rectangular  FOV  and 
1.37  x  1.37  x  1.5-mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices.  There  is  over-sampling  at  31%  in 
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the  slice  direction  to  reduce  aliasing  artifacts.  This  sequence  gave  excellent  image  contrast  for  the  prostate  and  its 
surroundings. 

3.2.  Creation  of  Images  for  Testing 

To  test  the  ability  of  the  slice-to-volume  (SV)  registration,  we  obtained  high-resolution  MRI  volumes  and  simulated 
iMRI  images  by  adding  noise  and  receive  coil  inhomogeneity.  We  compared  SV  registration  to  volume-to-volume  (VV) 
registration  results. 

We  simulated  iMRI  slice  images.  We  averaged  3  slices  together  to  simulate  an  iMRI  4-mm  thick  slice.  We  used  a 
homogenous  phantom  to  measure  the  SNR  of  iMRI  images  on  our  0.2  T  open  magnet  system  (Magnetom  Open,  Siemens 
Medical  Systems  in  Erlangen,  Germany).  We  used  simulated  iMRI  images  to  test  the  dependence  of  registration  on  noise 
levels.  Figure  2  shows  the  simulated  iMRI  slice  images. 

3.3.  Registration  Experiments 

We  acquired  high-resolution  MR  volumes  from  three  volunteers.  Each  has  two  pairs  of  volumes  for  registration.  We 
extracted  three  slices  from  one  volume  and  used  these  slices  to  simulate  an  iMRI  image.  We  tested  multiple  ways  to  acquire 
slices  for  SV  registration.  First,  we  used  transverse,  sagittal  and  coronal  slices  for  registration,  respectively.  The  objective 
was  to  optimize  the  slice  orientation  for  image  guidance.  Second,  we  extracted  slices  from  different  positions  such  as 
centered  at  the  prostate,  and  above  and  below  the  prostate  by  35  mm.  This  experiment  was  performed  to  investigate  the  range 
for  reliable  slice  registration.  Third,  we  tested  the  dependence  of  the  registration  on  noise  levels. 


Figure  2.  Simulated  iMRI  images.  Images  on  the  left,  (a),  (c)  and  (e),  are  the  high-resolution  MR  images  in  transverse, 
coronal  and  sagittal  planes,  respectively.  Images  on  the  right  are  corresponding  simulated  iMRI  images  with  SNR  -  8. 
The  images  are  of  volunteer  V2. 
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Figure  3.  SV  registration  using  slices  at  different  orientations.  The  error  metric  is  voxel  displacement  between  the  SV  and 
VV  results.  Plotted  are  mean  errors  as  well  as  maximums  and  minimums  that  show  the  spread  of  the  data.  SI,  S2,  and  S3 
refer  to  volunteers.  Each  volunteer  has  two  volume  pairs.  For  each  pair,  five  transverse  slices  centered  at  the  prostate  were 
extracted  from  one  volume  and  registered  to  the  other.  The  same  procedures  were  applied  to  coronal  and  sagittal  slices. 

3.4.  Evaluation  Methods 

To  test  the  program,  we  transformed  a  reference  volume  using  known  parameters  to  obtain  a  digital  phantom.  We 
extracted  slices  from  the  digital  phantom  and  registered  them  to  the  reference  volume.  Because  the  transformation  was 
known,  we  could  validate  the  performance  of  the  software. 

Our  standard  evaluation  method  was  to  compare  SV  and  W  registration.  The  VV  registration  accuracy  was  previously 
evaluated  to  be  on  the  order  of  one  voxel  (1.37  mm)  14.  We  defined  a  volume  of  interest  (VOI)  just  covering  the  prostate  and 
applied  the  transformations  obtained  by  VV  and  SV  registrations  to  voxels  within  the  VOI.  We  calculated  the  3D 
displacement  on  a  voxel-by-voxel  basis  over  the  two  transformed  VOIs.  The  mean  voxel  displacement  was  used  as  our 
metric  of  SV  registration  error. 


4.  RESULTS  AND  DISCUSSIONS 

More  than  300  SV  registration  experiments  were  performed  under  a  variety  of  conditions  expected  for  applications  in 
iMRI-guided  treatment  of  the  prostate.  First,  we  report  the  effect  of  slice  orientation  and  position  on  registration.  Second,  we 
describe  the  results  associated  with  image  noise.  At  the  end,  we  describe  some  details  on  the  algorithm  implementation. 
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Above  Prostate  Prostate  Center  Below  Prostate 


Figure  4.  SV  registration  using  transverse  slices  at  different  positions.  Relative  to  the  prostate,  five  iMRI  slices  each  were 
extracted  near  its  center,  *  35  mm  above  its  center,  and  ~  35  mm  below  its  center.  Other  details  are  given  in  Figure  3. 
Slices  at  the  prostate  center  work  best. 


4.1.  Slice  Orientation  and  Position 

Figure  3  shows  registration  results  for  single  slices  oriented  in  different  directions.  All  slices  intersect  the  prostate. 
Transverse  slices  give  the  lowest  average  error  across  the  three  volunteers,  0.39  ±  0. 19  mm.  The  reason  for  this  result  is  that  a 
transverse  slice  contains  more  anatomical  structures  than  do  slices  in  other  orientations.  A  transverse  slice  also  excludes  the 
bladder  and  portions  of  the  rectum  that  can  deform  and  create  inconsistent  matches  for  registration.  The  following  analyses 
are  all  based  on  transverse  slices. 

Figure  4  shows  registration  results  for  transverse  slices  at  different  heights  relative  to  the  prostate.  Slices  centered  on  the 
prostate  produced  the  best  results  with  the  displacement  error  always  being  less  than  1.0  mm.  On  the  one  hand,  slices 
centered  at  the  prostate  are  good  because  they  include  an  abundance  of  bony  structures  that  provide  adequate  information  for 
registration.  Slices  above  the  prostate  include  the  bladder,  which  can  deform  and  stress  the  registration  algorithm.  Slices 
below  the  prostate  mainly  contain  fatty  regions  from  the  hips  that  can  also  deform  and  have  less  information  for  registration. 

4.2.  Noise  Level  and  Robustness 

We  performed  preliminary  experiments  using  different  noise  levels  in  the  simulated  iMRI  images.  When  SNR  values 
were  30,  15  and  5,  registration  errors  were  always  less  than  0.9  mm  for  three  volunteers  in  all  conditions.  For  more  than  200 
registration  experiments,  the  algorithm  was  very  robust  and  never  failed  even  with  these  large  noise  values.  We  believe  the 
algorithm  to  be  insensitive  to  noise. 

4.3.  Program  Implementation 

We  implemented  the  registration  program  using  IDL  (Interactive  Data  Language,  Research  System  Inc.,  Boulder,  CO.). 
The  time  for  an  SV  registration,  typically  40  to  90  sec  on  a  Pentium  III,  800  MHz  CPU,  with  512  Mbytes  of  memory,  could 
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be  improved  using  optimized  C  code  rather  than  IDL.  The  simplex  optimization  method  worked  faster  than  the  Powell 
method  in  our  implementation.  This  result  is  contrary  to  our  previous  experience.1  We  think  that  this  phenomenon  occurred 
because  the  Powell  method  depends  on  the  order  that  parameters  are  optimized.  Because  we  do  not  require  preprocessing  to 
determine  initial  values,  there  are  no  clear  choices  for  which  parameters  to  optimize  first.  Each  call  to  the  Simplex 
optimization  for  restarts  or  optimization  at  a  finer  resolution  resulted  in  50  to  105  similarity  evaluations  before  the  tolerance 
value  (0.001)  was  reached. 

The  restarting  and  multi-resolution  features  are  important.  Using  normal  clinical  transverse  images  covering  the  prostate, 
the  algorithm  always  gave  very  nearly  the  same  transformation  parameters  (less  than  0.01  voxels  and  0.01  degrees)  using  a 
wide  variety  of  initial  guesses.  The  multi-resolution  approach  enabled  the  program  to  get  close  to  the  final  value  quickly 
because  of  the  reduced  number  of  calculations.  Althought  serious  motion  artifacts  may  stress  the  registration,  our  experience 
is  that  these  can  be  controlled  with  appropriate  subject  compliance  and  with  the  proper  acquisition  technique.  Organ  motion 
and  deformation  from  one  acquisition  to  the  next  are  other  factors  that  can  affect  the  performance  of  the  algorithm,  especially 
when  subject  is  in  a  different  position  or  there  is  rectal  and  bladder  filling.  14  Again,  our  experience  is  that  these  errors  can 
be  controlled.  14 


5.  CONCLUSION 

The  automatic  slice-to-volume  mutual  information  registration  algorithm  was  quite  robust  for  transverse  slice  images 
covering  the  prostate.  There  were  no  registration  failures  in  over  200  experiments  on  MRI  images  without  obvious  artifacts. 
The  registration  error  of  <  1.0  mm  should  be  sufficiently  accurate  to  aid  in  the  application  of  iMRI-guided  minimal  invasive 
thermal  ablation  of  prostate  cancer.  We  are  beginning  to  explore  these  applications  in  clinical  procedures  and  animal 
experiments. 
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